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SYSTEM APPLICATION TECHNIQUES FIG. 8 illustrates clock synchronization in a data acqui- 

USING TIME SYNCHRONIZATION sition and control system which includes a set of cards 

connected to a backplane. 


BACKGROUND OF THE INVENTION 
I. Field of Invention 


DETAILED DESCRIPTION 


„ . ^ tJ , FIG. 1 shows a distributed system 10 which includes a 

The present invention pertains to the field of systems. mastef ^ u xh&l dme vahies to a ^ 

More particularly, thts invention relates to system apphca- of ^ ^ nQdc M ^ ^ nodes 2Q _ 22 m 

turn techniques usmg time synchronization technology. in tercoiinected via a communication link 12. The master 

2. Art Background 10 node 14 includes a master clock 18 and the nodes 20-22 

Distributed control systems are commonly arranged as a include a set of slave clocks 30-32, respectively, 

collection of nodes which are interconnected via one or jh c mastcr no de 14 includes a traceable time source 16 

more network communication links. These network com- mal generates traceable time values. A traceable time value 

munication links may be packetized links such as Ethernet may DC defined as a time value which is derived from a 

or one or more of a variety of other packetized links that arc 15 standard time such as UTC time which was formerly known 

adapted to distributed control system applications. as Greenwich Mean Time (GMT). The mastcr clock 18 

Distributed control systems commonly benefit from pre- includes mechanisms that synchronize a master time value 

cise control of the timing at the distributed nodes. U.S. Pat. held in the mastcr clock 18 to the traceable lime values 

No. 5,566,180 of Eidson et. al. provides a method and obtained from the traceable time source 16. 

apparatus for providing precise control of timing in distrib- 20 The master clock 18 and the slave clocks 30-32 imple- 

uted nodes by synchronizing the local clocks in the distrib- mcm a synchronization protocol 100. According to the 

utcd nodes. In many applications, it may be desirable that the synchronization protocol 100, the master clock 18 and the 

time kept by the distributed nodes be traceable to a standard slave clocks 30-32 exchange packets via the communication 

time. Such may be useful in applications, for example, in \2 so that the slave clocks 30-32 synchronize to the 

which events in the distributed nodes must accurately occur 25 master time value held in the master clock 18. As a 

with respect to a particular date and/or time of day. consequence, traceable time values are accurately distrib- 

„ „„ „* mn ^ T uted to the nodes 20-22 using the synchronization protocol 

SUMMARY OF THE INVENTION 100 because ihc milS lcr dm, value to tbc m^ter clock 18 is 

A method and apparatus is disclosed for accurately dis- synchronized to the traceable time values from the traceable 

tributing traceable time values to a set of nodes in a system. time source 16. 

Each node includes a slave clock that synchronizes a slave In one embodiment, the synchronization protocol 100 and 

time value using a synchronization protocol. The system related mechanisms implemented in the master clock 18 and 

includes a traceable time source that generates a traceable the slave clocks 30-32 are those described in U.S. Pat. No. 

time value and a master node having a master clock that 35 5,566,180. For example, each of roe slave clocks 30-32 may 

synchronizes a master time value to the traceable time value include circuitry for adjusting its respective locally stored 

and that distributes the master time value to the slave clocks time value based upon computations of the sending and 

via the communication link. The nodes may be distributed receiving time of time data packets which arc transferred 

nodes or cards connected to a backplane. Also disclosed are over the communication link 12. The adjustment of a locally 

a variety of techniques for distributing the information ^ stored time value may be accomplished by implementing a 

associated with the synchronization protocol and a variety of local clock in each slave clock 30-32 as a counter driven by 

applications for the synchronized timing in the slave nodes. an oscillator with sufficient stability. The least significant 

Other features and advantages of the present invention few bits of the counter may be implemented as an adder so 

will be apparent from the detailed description that follows. that the increment on oscillator periods may be occasionally 

increased or decreased to effectively speed up or slow down 

BRIEF DESCRIPTION OF THE DRAWINGS the local clock in accordance with the results of the com- 

The present invention is described with respect to par- putation. 

ticular exemplary embodiments thereof and reference is In one embodiment, the traceable time source 16 is a 

accordingly made to the drawings in which: 8 lobal positioning system (GPS) receiver. In other 

FIG. 1 shows a distributed system which includes a 50 embodiments, other types of traceable time sources maybe 

master node that distributes traceable time values to a set of uscd ,ncludm g ratbo broadcasl *> mc ' & WV 

nodes; or alomic clodcs - 

FIG. 2 illustrates a node with a waveform generator that , The mas,c / ™* l * «" d thc ™dcs 20-22 may be any type 

derives a waveform period from the counter in a slave clock; of node in r ^ d^nbuted system 10. For example any one 

_ _ - ... . , ... . . . ,. 55 or more of the master node 14 and thc nodes 20-22 may be 

FIG. 3 .liustrates a master clock with traceable time in one a ^ ^ or an actuator nodc of an appUcation 

embodiment, node 0f fl combmat i on D f mcse m a distributed control 

FIG. 4 illustrates a technique for introducing traceable sys t em . Any one or more of the master node 14 and the 

lime into a system which includes a set of computer systems; Qodes 2 q_22 may be a computer system such as a personal 

FIG. 5 illustrates a technique for using one synchroniza- 60 computer, 

tion protocol to distribute the time values provided by a ^ communication link 12 may be implemented with 

different synchronization protocol; onc or morc of a varicty of communication mechanisms. In 

FIG. 6 shows a technique for distributing traceable time one embodiment, the communication link 12 is an Ethernet 

values to nodes which are connected to different sub-nets of communication network. In another embodiment, the com- 

a system; 65 munication link 12 is a LonTalk field-level control bus 

FIG. 7 illustrates one embodiment of a master/slave clock which is specialized for the process control environment. In 

in a boundary node; other embodiments, the communication link 12 may be 
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implemented with time division multiple access (TDMA) or 
token ring protocols to name only a few possibilities. 

The synchronization protocol 100, with or without trace- 
able time, may be used to provide periodic phase coherent 
signals at widely dispersed points which correspond to the 5 
physical locations of the nodes 20-22. For example, each of 
the nodes 20-22 may implement a schedule of times mark- 
ing desired signal events. Each node 20-22 compares these 
scheduled times to the local times provided by the corre- 
sponding slave clocks 30-32. When a scheduled event time 10 
matches the local time with appropriate accuracy, then a 
scheduled event is generated. The scheduled events may be 
sampling events wherein the nodes 20-22 include attached 
sensors. The nodes 20-22 may include waveform generators 
if analog signals are needed for events. In such a case, a 15 
match between local time and a time of a scheduled event 
triggers a waveform generator. 

FIG. 2 illustrates the node 20 with a waveform generator 
that derives a waveform period from the counter in the slave 
clock 30. The slave clock 30 includes a clock oscillator 160, 20 
a local clock 162, and a clock synchronization engine 164. 
The clock synchronization engine 164 obtains timing data 
packets from the communication link 12 and synchronizes a 
slave time value in the local clock 162 using the synchro- 
nization protocol 100. The local clock 162 provides a slave 25 
time value 184 to a comparator 170. The comparator 170 
compares the slave lime value 184 to an event time held in 
a register 168. The event time is provided by a computation 
engine 166. When the slave time value 184 matches the 
event time in the register 168 the comparator 170 triggers a 30 
periodic signal generator 172 which generates a waveform 
180. 

The period of the waveform 180 is controlled by a set of 
signals 182 from the local clock 162. The local clock 162 3S 
includes a counter portion and a clock adjustment portion. It 
is preferable that the signals 182 provide bits of the counter 
portion of the local clock 162 which are more significant 
than the lower order bits of the local clock 162 used for the 
clock adjustment portion. The periodic signal generator 172 4Q 
may be implemented using a digital phase locked loop 
which is driven by the signal 182, or using other well know 
techniques. 

The synchronization protocol 100, with or without trace- 
able time, may be used (o provide the basis for time division 45 
multiplex access (TDMA) communication or other temporal 
mutual exclusion communication protocols among the mas- 
ter node 14 and the nodes 20-22. The master node 14 and 
each node 20-22 may be allocated time slots the boundaries 
of which provide event triggers for TDMA communication. 50 
The event triggering techniques described above may be use 
to trigger time slot communication on the communication 
link 12 in each of the master node 14 and the nodes 20-22. 

The synchronization protocol 100, with or without trace- 
able time, may be used to provide an accurate time base for 55 
data acquisition and control in the system 10. The techniques 
disclosed above for providing phase coherent signals in the 
master nodes 14 and nodes 20-22 enables the establishment 
of a periodic time base for data acquisition and control. The 
generation of these phase coherent signal may be used to 6 q 
form the basis for a class of supervisory control and data 
acquisition (SCADA) systems. 

The techniques disclosed above for providing phase 
coherent signals enables the timing function to be distributed 
to the actual measurement or control nodes. This provides 65 
for improved accuracy, robustness and flexibility. As an 
example of flexibility, different portions of a SCADA system 
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using the techniques disclosed herein may be configured to 
run at different periodic or scheduled frequencies without 
any conflict of resources. This is in contrast to prior cen- 
tralized systems in which the management of two or more 
processes with different periodicity in general produces 
resource contention in the applications. This can cause 
timing jitter in the controlled processes. 

A distributed system which generates "time ticks" using 
present techniques for providing phase coherent signals is 
more accurate than prior systems. In addition, such a system 
is more robust in that the synchronization protocol 100 tends 
to keep the local clocks in sync even in the face of lost 
timing data packets. Moreover, the synchronization protocol 
100 consumes minimal amounts of network bandwidth in 
comparison to prior systems. 

FIG. 3 illustrates the master clock 18 in one embodiment. 
The master clock 18 includes a computation engine 40, a 
time-stamp latch 42, an adjustable clock 44, at time-stamp 
latch 46, and a time packet recognizer 48. The arrangement 
shown causes the time value in the adjustable clock 44 to be 
controlled by the traceable time source 16 and causes the 
time values held in the slave clocks 30-32 to be controlled 
by the time value in the adjustable clock 44. 

The traceable time source 16 generates an updated trace- 
able time value 50 and a series of pulses 52 which in one 
embodiment occur once per second. The pulses 52 continu- 
ously cause the time-stamp latch 42 to latch time values 
from the adjustable clock 44. The computation engine 40 
compares the updated traceable time value 50 to the time 
values from the lime-stamp latch 42 and issues a set of clock 
adjustment signals 54 that cause the adjustable clock 44 to 
move toward and eventually match the updated traceable 
time value 50. 

The compulation engine 40 issues the clock adjustment 
signals 54 to cause the adjustable clock 44 to either speed up 
or slow down or maintain rate or reload with a new time 
value. In one embodiment, the adjustable clock 44 is imple- 
mented as a counter driven by an oscillator with an adder 
that adds either 0, 1, or 2 to the counter during each period 
of the oscillator. If the time value held in the time-stamp 
latch 42 is less than the updated traceable time value 50 then 
the computation engine 40 issues the clock adjustment 
signals 54 to cause a 2 to be added to the counter of the 
adjustable clock 44. If the time value held in the time-stamp 
latch 42 equals the updated traceable time value 50 then the 
computation engine 40 issues the clock adjustment signals 
54 to cause a 1 to be added to the counter of the adjustable 
clock 44. If the time value held in the time-stamp latch 42 
is greater than the updated traceable time value 50 then the 
computation engine 40 issues the clock adjustment signals 
54 to cause a 0 to be added to the counter of the adjustable 
clock 44. If the difference between the time value held in the 
time-stamp latch 42 and the updated traceable time value 50 
is greater than a predetermined value then the computation 
engine 40 uses the clock adjustment signal 54 to reload the 
adjustable clock 44. 

The time-stamp latch 46 obtains time values from the 
adjustable clock 44. The time packet recognizer 48 in 
response generates time data packets and transfers them via 
the communication link 12 to cause the slave clocks 30-32 
to synchronize to the time value held in the time-stamp latch 
46. This is in accordance with the synchronization protocol 
100. 

FIG. 4 illustrates a technique for introducing traceable 
time into a system 70 which includes a set of computer 
systems 60-62 that participate in a synchronization protocol 
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72 which differs from the synchronization protocol 100. In source 144 and a master clock 146. The traceable rime 

one embodiment, the synchronization protocol 72 imple- source 144 provides traceable time values to the master 

mented by the computer systems 60-62 is the network time clock 146 and the master clock 146 distributes the traceable 

protocol (NTP). time values to a set of slave clocks 140-142 in the nodes 

The computer systems 60-62 may be personal computers s 120-122 using the synchronization protocol 100. 

or workstations or a combination of these. In accordance ^ syslem 150 mcmdcs a boundary node 142 coupled 

with NTP, the computer systems 60-62 periodically bctwccn lhc communication links 110-112. The boundary 

exchange messages containing the Local syslem clock lime no(Je U2 mdudes a masler/slave ckjck 152 . The master/ 

:^r~a" 10 «j™ I 52 bc hl T *7^Jl r 11 T 

♦ i i ■ *L enjrT „„„ 10 chromzes to traceable time values distributed by the master 

system clocks in the computer systems 60-62 converge. nn, i jj .- 

1 ~ , , Jen . i • i ik» mto clock I 46 via the communication link 110. In addition, the 

The computer system 60 not onlyj^me^ to NTP masler/slave dock 152 behavcs like a master clock in mat it 

protocol, but also mcludes a slave clock 66 which synchro- . » ^ ... . . , , . 

r . .„ , k « . , t . .„i,,„ u„ m , t .- r m turn distributes the traceable tune values to a set of slave 

nizes to the traceable time values distributed by the master ^„ . , . rt , . 

clock 18 using the synchronization protocol 100. The soft- clocks 150-152 m the nodes 130-132 via the communica- 

warc layers of the NTP protocol in the computer system 60 ^an link 112. 

obtain time values from the slave clock 66 raiher than the FIG. 7 illustrates one embodiment of the master/slave 

system clock of the computer system 60. In addition, the clock 152. The master/slave clock 152 includes an adjust - 

slave clock 66 remains synchronized to the master clock 18 able clock 254 along with a time packet recognizer 250 and 

rather than converging with the NTP protocol as do the a latch 252 for the slave side on the communication link 110 

system clocks of the computer systems 61-62. As a result, 20 and a time packet recognizer 258 and a latch 256 for the 

the system clocks of the computer systems 61-62 eventually master side on the communication link 112. A computation 

converge to the time value in the slave clock 66 which is engine 260 performs the computations for both the master 

traceable time. and slave sides of the master/slave clock 152. 

The slave clock 66 may be implemented on a standardized The operation of the slave side of the master/slave clock 

interface card for the computer system 60. The same inter- 152 is as follows. The master clock 146 transfers a timing 

face card may contain both the slave clock 66 and the data packet (a timing event) via the communication link 110 

elements needed for communication on the communication and then transfers a data packet containing a traceable lime 

link 64. For example, one interface card may contain the value for tbe timing event (the supplemental information for 

slave clock 66 and an Ethernet interface if the communica- 3Q the timing event) via the communication link 110. In 

lion link 64 is Ethernet. response to the timing event, the time packet recognizer 250 

In one embodiment, the master node 14 is a computer causes the latch 252 to latch the local time in the adjustable 

system such as a personal computer or workstation. In such clock 254. The computation engine 260 receives the supple- 

an embodiment, the host processor of the master node 14 mental information and computes the difference between the 

may perform the functions associated with the computation 35 time value in the latch 252 and the traceable time value from 

engine 40. the supplemental information and then adjusts the adjustable 

FIG. 5 illustrates a technique for using the synchroniza- clock 254 accordingly. This causes Ihe adjustable clock 254 

tion protocol 100 to distribute the time values obtained with to sync to the traceable time values distributed by the master 

the synchronization protocol 72. A system 80 is shown clock 146 

including a set of computer systems 90-92 that participate in 40 The operation of the master side of the master/slave clock 

the synchronization protocol 72 which may be NTP via a 152 is as follows. The computation engine 260 periodically 

communication link 82. The computer system 90 includes a generates a timing data packet and transfers it via the 

master clock 86 which participates in the synchronization communication link 112 to the slave clocks 150-152. In 

protocol 100 via the communication link 82 with a node 84 response to a timing data packet, the time packet recognizer 

having a slave clock 88. The computer system 90 obtains a 45 258 causes the latch 256 to latch the local time in the 

time value from its system clock and provides it to the adjustable clock 254. The computation engine 260 then 

master clock 86. The master clock 86 distributes this time obtains the latched time value from the latch 256 and 

value to the slave clock 88 using the synchronization pro- transfers it as supplemental information via the communi- 

tocol 100. cation link 112 to the slave clocks 150-152. This enables the 

The master clock 86 may be implemented on a standard- 50 computation engines in the slave clocks 150-152 to sync up 

ized interface card for the computer system 90. The same to traceable time values of the adjustable clock 254. For 

interface card may contain both the master clock 86 and the example, the slave clock 150 uses the difference between its 

elements needed for communication on the communication reception time of a timing data packet and the traceable time 

link 82. The host processor of the computer system 90 may value provided by the corresponding supplemental informa- 

perform the functions associated with the compulation 55 to adjust its local adjustable clock, 

engine of the master clock 86. Alternatively, the masler The computation engine 260 may render a determination 

clock 86 may contain a computation engine which obtains of which is the master side and which is the slave side of the 

syslem time values from the host processor of the computer master/slave clock 152 in response to information provided 

system 90. by the clocks attached to each communication link 110-112. 

FIG. 6 shows a technique for distributing traceable time 60 Tb^ information may include the accuracy of the clocks, 

values to nodes which are connected to different sub-nets of whether a clock provides traceable time values either 

a system 150. One sub-net of the system 150 includes a set directly or as a slave to a traceable time source, and whether 

of nodes 120-122 coupled to a communication link 110 and clocks are themselves two-sided clocks of a boundary node, 

another sub-net of the system 150 includes a set of nodes A master/slave clock in a boundary node in general has 

130-132 coupled to a communication link 112. $5 one slave side and n master sides for linking up to n 

Traceable time values are introduced into the system 150 additional subnets. Each slave side and each of the n master 

using a master node 140 which includes a traceable time sides have corresponding time packet recognizers and 
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latches and the same computation engine may be shared somewhat more complex manner. For example, information 

among them. The slave side is a slave on the subnet which may be carried on the communication links 220-223 using 

has the best source of time according to a preselected criteria a signal packet. A start of frame portion of the signal packet 

such as accuracy or any of the other criteria set forth above. may indicate the timing event and the supplemental infor- 

FIG. 8 shows a data acquisition and control system 200 5 malum may be encoded in the remainder of the signal 

which includes a set of cards 210-213 connected to a P^ 1 ' ^"ffTn ?„ RS232 im P lementaUoD of 

backplane 202. The cards 210-213 each include circuitry for communicator .links 220-223. 

providing synchronized clocks using the synchronization . In rac embodimcjl ^card213 is the master card which 

* i iaa j • u j e l. includes a master clock that distributes time values to slave 

protocol 100 and mchide circuitry for generating phase cloc ^ m me ^ 2l0 _2l2 and to slave clocks in nodes that 

coherent signals using techniques outline above. w m {Q a c^^tion link 204. These time 

The synchronization protocol 100 requires that two basic valucs may ^ obtained from a host processor connected to 

types of information be communicated among the cards the backplane 202 which implements the NTP protocol. 

210-213. The basic types are a timing event and a set of ln mother embodiment, the card 213 includes a master 

supplementary information associated with the timing event. c i oc k that distributes time values to slave clocks in the cards 

For example, a timing data packet is a timing event. If 15 210-212 and includes a slave clock that synchronizes to a 

traceable time is provided by a GPS source then the timing master clock in a node connected to the communication link 

data packets usually occur at one per second. 204. The node connected to the communication link 204 and 

The supplemental information for a timing event is an having the master clock may include a traceable time source 

interpretation of the timing event. For example, if a timing such as GPS as described above. The communication link 

event is associated with a GPS source then the supplemental 20 204 may be a network communication link 204 such as 

information for the timing event is the UTC time at which Ethernet. The card 213 may also serve as a port to the 

the timing event was generated. If a timing event is associ- communication link 204 for a host processor connected to 

ated with a master clock then the supplemental information the backplane 202. 

for the timing event is the time at which the node having the The computations needed for the synchronization proto- 

master clock generated the timing event. 25 col 100 may be performed by computation circuitry imple- 

The cards 210-213 communicate timing events using a mented on the cards 210-213. Alternatively, the computa- 

set of communication links 220-223 arranged as a daisy u ons needed for the synchronization protocol 100 may be 

chain. Each of the cards 210-213 includes a daisy chain performed by a host processor connected to the backplane 

input port (IN) and a daisy chain output port (OUT). Each 3Q 202. 

of the cards 210-213 receives timing events on its IN port The backplane 202 may be that of a personal computer or 

and issues timing events on its OUT port. If a card docs not workstation or a specialized system. The backplane 202 may 

detect any timing events on its IN port then it can assume be a standard backplane such as PCI, VME, or ISA to name 

that it is the master of the data acquisition and control system a few examples. 

200. Alternatively, the master of the data acquisition and 35 A communication link for communicating the timing 

control system 200 may be configured by a host processor events and the supplemental information for the synchroni- 

connected to the backplane 202. The one of the cards zation protocol 100 may be implemented as an additional 

210-213 that is master issues timing events at an appropriate trace on the backplane 202. However, this may require a 

rate. Each of the remaining ones of the cards 210-213 that modification to an existing backplane standard which may 

is a slave card passes each timing event from its IN port to AQ not be practical. In addition the communication link for 

its OUT port without significant delay. communicating the timing events and the supplemental 

In one embodiment, the communication links 220-223 information may be a packetized network such as Ethernet 

are signal lines. In another embodiment, the communication which is connected to each of the cards 210-213. This may 

links 220-223 are fiber optic links. The cards 210-223 may be loo expensive to implement for a backplane system, 

include LEDs and photo-diodes so that when each card 45 The foregoing detailed description of the present inven- 

210-223 is inserted in the backplane 202 a daisy chain input tion is provided for the purposes of illustration and is not 

and output link is formed. In other embodiments, the com- intended to be exhaustive or to limit the invention to the 

munication links 220-223 may be RS232 links or network precise embodiment disclosed. Accordingly, the scope of the 

communication links such as Ethernet. present invention is defined by the appended claims. 

The one of the cards 210-213 having the master clock 50 What ^ claimed is: 

generates the supplemental information and may be referred I A system, comprising: 

to as the master card. The remainder of the cards 210-213 a set of nodes coupled to a communication link and each 

receive the supplemental information and may be referred to having a slave clock that synchronizes a slave time 

as slave cards. In one embodiment, the cards 210-213 value using a synchronization protocol; 

communicate the supplemental information using commu- 55 traceable time source that generates a traceable time 

nication paths on the backplane 202 since the supplemental value; 

information is less time critical than the timing events and master node coupled to the communication link and 

therefore the delays associated with communication on the having a master clock that synchronizes a master time 

backplane 202 may be tolerable. This embodiment allows value to the traceable lime value and that distributes the 

simple and low cost implementations of the communication 60 master time value to the slave clocks via the commu- 

links 220-223. For example, the communication links nication link using the synchronization protocol. 

220-223 may be signal lines lhat carry CMOS signals which 2. The system of claim 1, wherein the traceable time 

communication timing events. source is contained in the master node. 

in an alternative embodiment, the cards 210-213 com- 3. The system of claim 1, wherein the traceable time 

municate the supplemental information using the commu- 65 source is a GPS receiver, 

nication links 220-223. This embodiment may require that 4. The system of claim 1, further comprising a computer 

the communication links 220-223 be implemented in a system coupled to the communication link and having a 
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slave clock which synchronizes to the master time value via 
the communication link using the synchronization protocol, 
the computer system executing a set of software for syn- 
chronizing a system clock of the computer system to a set of 
other system clocks in a set of other computer systems s 
coupled to the communication link such that the computer 
system substitutes a time value from the slave clock for a 
time value in the system clock for use in synchronizing the 
other system clocks in the other computer systems. 

5. The system of claim 4, wherein the communication link 10 
enables communication using a packelized network com- 
munication protocol. 

6. The system of claim 5, wherein the slave clock of the 
computer system and circuitry for the packetized network 
communication protocol are both implemented on an inter- 15 
face card for the computer system. 

7. A system, comprising: 

a set of nodes coupled to a communication link and each 
having a slave clock that synchronizes a slave time 
value using a synchronization protocol; 20 

computer system coupled to the communication link and 
executing a set of software for synchronizing a system 
clock of the computer system to a set of other system 
clocks in a set of other computer systems coupled to the 
communication link, the computer system having a 25 
master clock that synchronizes a master time value to 
the system clock and that distributes the master time 
value to the slave clocks via the communication link 
using the synchronization protocol. 

8. The system of claim 7, wherein the communication link 30 
enables communication using a packetized network com- 
munication protocol. 

9. The system of claim 8, wherein the master clock and 
circuitry for the packetized network communication proto- 
col are implemented on an interface card for the computer 35 
system. 

10. A system, comprising: 

first sub-net having a set of first nodes coupled to a first 
communication link, each first node having a slave 4(J 
clock that synchronizes a slave time value using a 
synchronization protocol on the first communication 
link; 

second sub-net having a set of second nodes coupled to a 
second communication link, each second node having 45 
a slave clock that synchronizes a slave time value using 
the synchronization protocol on the second communi- 
cation link; 

traceable time source (hat generates a traceable time 
value; 50 

master node coupled to the first communication link and 
having a master clock that synchronizes a master time 
value to the traceable time value and that distributes the 
master time value to the slave clocks via the first 
communication link using the synchronization proto- 55 
col; 
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boundary node coupled to the first communication link 
and having a master/slave clock that synchronizes to 
the master time value distributed by the master clock 
using the synchronization protocol on the first commu- 
nication link and that distributes the master time value 
to the slave clocks of the second sub-net via the second 
communication link using the synchronization proto- 
col. 

U. The system of claim 10, wherein the traceable time 
source is contained in the master node. 

12. The system of claim 10, wherein the traceable time 
source is a GPS receiver. 

13. A system, comprising: 

master node coupled to a communication link and having 
a master clock that distributes a master time value via 
the communication link using a synchronization pro- 
tocol; 

a set of nodes coupled to the communication link and each 
having a slave clock that synchronizes a slave time 
value to the master time value using the synchroniza- 
tion protocol, wherein one or more of the nodes 
includes circuitry for generating a phase coherent peri- 
odic signal in response to the slave time value. 

14. The system of claim 13, wherein the phase coherent 
periodic signal is used for synchronizing TDM A communi- 
cation on the communication link. 

15. The system of claim 13, wherein the phase coherent 
periodic signal is used for a lime base in data acquisition and 
control. 

16. The system of claim 13, further comprising a traceable 
time source that generates a traceable time value such that 
the master clock synchronizes the master time value to the 
traceable time value. 

17. A data acquisition and control system, comprising: 

a set of slave cards connected to a backplane, each slave 
card having a slave clock for use with a synchroniza- 
tion protocol which includes at least one timing event 
and a set of supplemental information associated with 
the timing event; 

master card connected to the backplane and having a 
master clock for use with the synchronization protocol; 

daisy chain communication link among the master card 
and the slave cards such that the master card commu- 
nicates the timing event to the slave cards using the 
daisy chain communication link and communicates the 
supplemental information to the slave cards using a 
communication path on the backplane. 

18. The data acquisition and control system of claim 17, 
wherein the master card is coupled to a network communi- 
cation link which is connected to one or more nodes each of 
which includes a slave clock such that the master clock in 
the master card distributes the timing event and the supple- 
mental information to the nodes via the network communi- 
cation link. 

***** 
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ABSTRACT 


A distributed synchronization system for use in each node of 
a distributed asynchronous telecommunications network 
system that continually monitors and controls the flow of 
data through an implementing node to prevent dataflow 
errors due to phase and frequency differences in source and 
destination nodal clocks, and to control inter-nodal network 
latency so as to support the transmission of synchronous 
data. A synchronization data FIFO buffers predetermined 
fields or portions of fields of a unique frame packet received 
from a source node before retransmission to a destination 
node on the network. The frame packet includes a frame 
synchronization field indicating the beginning of a new 
frame packet; a payload field containing valid data; and a 
dead zone field providing bandwidth during which the 
present invention performs synchronization functions. A 
frame synchronization subsystem, implemented in a desig- 
nated master node, guarantees that a frame is released at (he 
beginning of an independently-determined frame regardless 
of network latency. A word resynchronization subsystem 
manages the flow of data through the data FIFO of each 
non-master node, receiving and storing data at the source 
node's clock rale and transmitting the data according to its 
own clock, thereby guaranteeing the efficient receipt and 
transmission of data between asynchronously* 
communicating nodes. 

6 Claims, 11 Drawing Sheets 
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DISTRIBUTED NETWORK 
SYNCHRONIZATION SYSTEM 

CROSS-REFERENCES TO RELATED 

APPLICATIONS AND PATENTS s 

The following commonly-owned patents and applications 
are related to the present invention and are incorporated by 
reference by this and other references: 

U.S. Pat. No. 5,544,163, entitled "Expandable Telecom- 
munications System " naming as inventor Robert P. 10 
Madonna; 

U.S. Pat. No. 5,426,694, entitled 'Telecommunications 
Switch Having Programmable Network Protocols and 
Communications Services," naming as inventor Mark 
P. Hebert; 15 

U.S. patent application Ser. No. OS/566,414, entitled 
"Telecommunications Switch Having A Universal 
Applications Program Interface," naming as inventor 
Mark P. Hebert, filed on Nov. 30, 1995; and ^ 

U.S. Pat. No. 5,349,579, entitled "Telecommunications 
Switch With Programmable Communications 
Services," naming as inventors Robert P. Madonna and 
Kevin C. Kicklighter. 

BACKGROUND OF THE INVENTION 25 

1. Field of the Invention 

This present invention relates generally to the field of 
telecommunications and, more specifically, to a distributed 
synchronization system for maintaining word and frame 30 
synchronization among asynchronous nodes of a distributed 
telecommunications system. 

2. Related Art 

Generally, a distributed network system includes a physi- 
cal carrier transport system that transports data between 35 
geographically-distributed nodes. The network may take 00 
any one of a number of architectural forms such as a bus or 
ring. In a bus topology, a coaxial cable or an optical fiber is 
commonly used as the physical carrier. Ethernet, for 
example, uses a coaxial cable as its transmission medium. In aq 
the case of a ring, the physical medium may be a twisted- 
wire pair, coaxial cable, or optical fiber. In a ring topology, 
nodes serve as repeaters, receiving and re-transmitting 
incoming information. 

Various approaches have been developed to avoid con- 45 
flicts between nodes using a shared medium in a network. 
For example, in one common technique, a token-passing 
distributed-control scheme is used, where permission to 
transmit is passed sequentially from one node or station to 
another by means of a "token," a specified bit or flag set in 50 
an information frame, or a specifically-defined control 
frame. Alternatively, a node currently engaged in transmit- 
ting over the medium may pass control to the next node upon 
conclusion of its transmission by setting a token bit in its 
transmitted frame. A node recognizing the token bit is then 55 
free to transmit its own information if it so desires. Thus, 
multiple tokens may simultaneously exist on the medium. In 
another conventional approach, a currently-transmitting 
node passes control to the next node (i.e., issues a token) 
only on the return of its own frame. In this case, there is only 60 
one token available at any one time, simplifying manage- 
ment of the network. Both bus and ring topologies may be 
used in conjunction with the token-passing approach. In the 
case of a bus, nodes are numbered in succession to permit 
unique identification of which node may next receive the 65 
token. In the case of a ring, the "next" node is implicit in the 
direction of transmission. 
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In the simplest mode of operation, each node on the ring 
receives each frame packet and then passes it on (retransmits 
it) to its neighbor. If a particular node recognizes the packet 
destination address as being its own, that node copies the 
frame in addition to retransmitting it. The original transmit- 
ting source node takes its own frame off the ring as it returns 
from one cycle around the ring. In the single-token 
procedure, a sending node passes control to the next node by 
issuing a token after receipt of its own frame packet. A node 
with a frame to transmit must wait until it receives the token 
before transmitting. 

The time to transfer data from a source node to a desti- 
nation node is typically used as a measure of network 
performance. The transfer time is dependent upon a number 
of factors, a significant one of which is ring latency or delay. 
There are two major contributors to ring latency: the propa- 
gation delay required for a frame packet to cycle once 
around the ring; and the delay required to retransmit a frame 
packet at each node on the ring. In general, reduced ring 
latency results in better network performance. 

The effect of ring latency is compounded by the increased 
bandwidth capabilities provided in modem high-speed fiber 
optic systems. As high speed networks become faster due to 
the packing of bits closer together in the fiber, the time it 
takes for a single bit to traverse the fiber stays essentially the 
same. Thus, for example, it may take approximately the 
same time to exchange a message between applications on 
a high speed fiber optic network, which may be capable of 
operating at 2 Gb/s, as it does over a 10 Mb/s Ethernet 
network. However, the increased capability of the fiber optic 
network to send more bits per unit time, as well as the 
increased capability of nodes to perform more instructions 
per unit time, results in an increase in the relative delay in 
the network. That is, the number of instruction cycles that a 
source node must wait for a reply to its message increases as 
the node's CPU cycle time decreases. As a result, ring 
latency is becoming the largest contributor to the reduction 
of performance in distributed network systems. 

This problem is exacerbated in widely distributed network 
systems since propagation delay increases with distance. As 
(he nodes in a distributed network become more geographi- 
cally distributed, the number of instruction cycles that a 
source node must wait for its packet to return, or for an 
answer to its message, increases. In addition, as the node- 
lo-node distance increases in a geographically distributed 
network system, the propagation delay, and thus ring 
latency, becomes unpredictable. The unpredictability of dis- 
tributed network systems is particularly problematic when 
the network is required to carry synchronous data such as 
pulse coded modulation (PCM) data commonly used in 
telecommunications networks. The unpredictable arrival of 
frame packets prevents the receiving node from accurately 
identifying the divisions between PCM samples, thereby 
inhibiting the transfer of synchronous data through the 
asynchronous network. 

Furthermore, to ensure proper reception of information 
over a distributed network, local timing signals (i.e., clock 
signals) controlling a given destination node must be pre- 
cisely matched to those of the source node. However, despite 
being designed to operate at the same frequency, timing 
variations inevitably exist among network components. 
High frequency variations, referred to as jitter, arc typically 
reduced to manageable levels through the use of jitter filters 
in each node. Low frequency variations, referred to as 
wander, are typically dealt with through the use of buffers 
located within the nodes of the network. Specifically, these 
buffers store a small amount of data, allowing it to build up 
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or be drained by small-magnitude wander without data loss Specifically, the present invention includes a syncbroni- 

or errors. When wander exceeds the capacity of the buffers, zation data first-in-first -out (FIFO) memory for storing pre- 

they either repeat (i.e., underflow) or discard (i.e., overflow) determined fields or portions of fields of a unique frame 

blocks of data to compensate for differences in timing packet. The frame packet includes a frame synchronization 

between the source and destination nodes. Underflow and 5 field which marks the beginning of the frame packet; a 

overflow conditions, generally referred to as slip, typically payload field containing valid data; and a dead zone field 

result in errors within the network. For example, in a voice which the present invention utilizes to perform synchroni- 

circuit, slip may appear as popping or clicking sounds, zation functions. A frame synchronization subsystem of the 

whereas in data transmissions, slip is manifested by the loss present invention, implemented in a designated master node, 

of data. Very large buffers can reduce the probability of such 10 operates such that a frame is released at the beginning of an 

errors, but they increase the delay through the network. independently-determined frame cycle regardless of the ring 

Delay is undesirable, so buffer size is generally minimized. latency of the network. 

Various techniques have been developed to maintain A word resynchronizalion subsystem manages the flow of 

network synchronization and avoid such loss of data. For data through the data FIFO of each non-master node, receiv- 

example, conventional synchronization techniques often 15 ing and storing the data at the source node's clock rate and 

require transmission of timing information through the net- re-transmitting the data at its own clock rate. Thus, the word 

work along with the data. A clock recovery system residing resynchronization subsystem controls the operation of the 

in a destination node uses the transmitted timing information synchronization data FIFO to effectively absorb any phase 

to recover the frequency of the source node's clock and to difference between the clocks of a source node and a 

generate a transmit clock having a frequency at which the 20 destination node implementing the invention. A write 

destination node transmits the data to a destination user controller, operating at the source node clock rate, and a read 

process. In addition, the recovered clock and data are controller, operating at the destination node clock rate, 

provided to other nodes in the network. Regardless of the asynchronously manage the passage of data through the data 

recovery technique, each node employs a phase-locked loop FIFO to maintain the level of data in the FIFO within an 

or other feedback circuitry that varies around the source 25 optimal range. During the receipt of a predetermined portion 

node's clock frequency, continually adjusting to maintain of a frame transmission which contains no valid data, the 

lock on that frequency. This continual adjustment around the FIFO read and write controllers may temporarily suspend 

desired frequency causes jitter. As each subsequent node read and/or write operations from/to the FIFO to maintain 

attempts to recover the clock, the jitter from all previous we data level within the optimal range for efficient data 

recovery attempts is accumulated. Eventually, this accumu- 30 transfer across the network. 

lated jitter may become too large, thereby resulting in data Advantageously, the word resynchronization subsystem 

joss. of the present invention prevents data FIFO overflow and 

Another drawback to conventional clock recovery sys- underflow conditions from arising, thereby ensuring sub- 
terns is that they are based upon the assumption that identical stantially error-free transmission through the implementing 
network reference clocks are provided to the source and 35 network node. A significant feature of the word resynchro- 
destination nodes. This is often not the case in nation subsystem is its anticipatory nature, enabling it to 
geographicaUy-distributed telecommunications systems. It compensate for phase differences between clock signals 
is not uncommon for each portion of a geographical- before data is lost. Another feature provided by the invenUon 
distributed telecommunications network to be synchronized is the ability to momentarily control cither or both the 
to a different reference clock. Although those local clocks 40 readin fi ™ d wrilin S of data from and 10 ™ F0 10 recover 
may be referenced to stratum 1 clocks, they may exhibit a & om clock deviations without loss of data and without 
phase difference over lime that continues to increase until a caus »>g substantial delays in the network, 
slip in inter-nodal communications occurs. Moreover, if a The frame synchronization subsystem, on the other hand, 
network element such as a digital cross connect fails, certain buffers only the payload and frame synchronization fields of 
network nodes may lose their reference clock. These nodes 45 toe frame packet, which are held until the beginning of a 
must then utilize their internal clocks, resulting in an frame cycle, as determined by the occurrence of a local 
increased loss of data due to the difference in phase and frame synchronization pulse. In response to that pulse, the 
frequency between such nodes' internal clocks and the frame synchronization subsystem transmits the frame syn- 
refcrcncc clocks chronizalion and payload fields, onto which this subsystem 

What is needed, therefore, is a means for enuring that the 50 a PP eods a new Ic^Uy-determined dead zone to create a 

ring latency in a distributed network system is reliably frame P*<* et thai is precisely one frame m length. This 

controlled so as to support the transmission of synchronous W ci of Present invention advantageously ensures that 

data. In addition, the system must be capable of compen- every non-master node receives a frame packet at predeter- 

sating for differences between source and destination nodes' „ mmcd intervals of time. In turn, such predictable and 

clocks without loss of data and without causing excessive 55 P« ,odlc receipt of frame packets enables the rece.vmg 

delays in the transmission of information across the net- 10 P/ ecisel y f m,ne . th ' boundaries between syn- 

wor j^ chronous data samples contained within the frame packets. 

Furthermore, the frame synchronization subsystem auto- 

SUMMARY OF THE INVENTION maucally adjusts the amount of data buffered in the master 

60 node, dynamically self-adjusting to changes in the ring 

In brief summary, the present invention provides a dis- latency which may be caused, for example, by the addition 

tributcd synchronization system for use in connection with ( or deletion) of nodes to the network. 

a distributed, asynchronous, telecommunications network ^ _ r __„, rv „^ T nr .„ tr . A „ IIVI ^ 

system that continually monitors and controls the flow of BRIEF DESCRIPTION OF THE DRAWINGS 

data through an implementing network node so as to prevent 65 The above and further advantages of the invention may be 

dataflow errors due to phase and frequency differences in better understood by referring to the following description in 

source and destination nodal clocks. conjunction with the accompanying drawings in which: 
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FIG. 1 is a block diagram of an expandable telecommu- 
nications system which employs a ring-type inter-nodaJ 
network to transfer information between nodes, all of which 
is constructed in accordance with a preferred embodiment of 
the present invention; 

' FIG. 2 is a block diagram of one type of programmable 
switching node that may be used in the telecommunications 
system of FIG. 1; 

FIG. 3 is a block diagram of the nodal switch incorporated 
in the switching node illustrated in FIG. 2; 

FIG. 4 is a block diagram of the distributed synchroni- 
zation system of the present invention; 

FIG. 5 is a block diagram showing the novel frame 
structure utilized by.thc distributed synchronization system 
of the present invention; 

FIG. 6 is a block diagram showing the word resynchro- 
nization subsystem of the distributed synchronization sys- 
tem of the present invention; 

FIG. 7 is a state diagram illustrating the functions per- 
formed by the write controller of the word resynchronization 
subsystem of the present invention; 

FIG. 8 is a state diagram illustrating the functions per- 
formed by the read controller of the word resynchronization 
subsystem of the present invention; 

FIG. 9 is a block diagram showing the frame synchroni- 
zation subsystem of the distributed synchronization system 
of the present invention; 

FIG. 10 is a state diagram illustrating the functions 
performed by the write controller of the frame synchroni- 
zation subsystem of the present invention; and 

FIG. 11 is a state diagram illustrating the functions 
performed by the read controller of the frame synchroniza- 
tion subsystem of the present invention. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 
A. System Environment 

FIG. 1 shows a large capacity, expandable, rally program- 
mable telecommunications switching network system 100. 
The network system 100 includes a series of programmable 
nodes 102 interconnected by a ring-architecture, inter-nodal 
network 110. The programmable nodes include a master 
node 102a, programmable switching nodes 1026, I02d, and 
a voice processing resources node 102c. A host link 114 
connects node 1026 in communicating relationship with a 
host computer 104. Nodes 102a, 102c and 102rf may be 
controlled by host computer 104, whether by additional host 
links to such nodes, by passing control information over 
inter-nodal network 110 or by separate host devices. 
Although only a single host computer 104 and host link 114 
are shown for purposes of improved clarity, use of a local 
area network (LAN) to provide host/node communications 
permits multiple hosts to control the system 100 (or parts 
thereof) by configuring each host as a "client" and each node 
as a "server." 

The nodes may perform any number of functions. For 
example, nodes 1026 and 102d are switching nodes and 
include desired network/line interfaces for connection, 
respectively, with a public switched telephone network 
(PSTN) or a private network 106 and 118, respectively. The 
term "private network" is intended in a broad sense to refer 
to any network or line or other interface other than the 
PSTN. Network/line interfaces 108, 116 may terminate 
either digital networks or analog trunks/lines, or combina- 
tions of both types. 

Node 102a is nominally designated a "master node," the 
significance of which is described below. As noted below, 
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any of nodes 1024-1024 may be configured as the active 
master node. However, at any given time, there may be only 
one active master node. 

Inter-nodal network 110 provides for high speed, high 

5 bandwidth digital communications among nodes 
I02o-102d. Inter-nodal network 110 may be implemented 
using one or more fiber optic rings which enable each of the 
nodes to exchange packetized information with each other 
node served by network 110. Inter-nodal network 110 may 

10 also be implemented with any of a variety of other types of 
communications networks, including Ethernet or other types 
of LANs, wireless communications networks or the PSTN 
(ATM/SONET). Using the PSTN for inter-nodal network 
110, for example, permits the nodes to be geographically 

is distributed over large areas. Furthermore, other inter-nodal 
network topologies, such as a bus topology, are contem- 
plated by the present invention. 

The overall operation of system 100 is controlled by host 
104, which is commonly implemented with a personal 

20 computer (PC), workstation, or other computer on which a 
user's application software runs. Host 104 and node 1026 
exchange messages over host link 114. Such messages are 
typically used to configure the node as well as direct call 
processing functions such as making connections and pro- 

25 viding communications services (i.e., tone detection, tone 
generation and conferencing). Descriptions of exemplary 
programmable network protocols and communications ser- 
vices supported by nodes 102, as well as the development of 
such protocols, may be found in commonly owned U.S. Pal. 

30 No. 5,426,694 to Mark P. Hebert, entitled "Telecommuni- 
cations Switch Having Programmable Network Protocols 
and Communications Services," and U.S. patent application 
of Mark P. Hebert, entitled "Telecommunications Switch 
Having A Universal Applications Program Interface," filed 

35 on Nov. 30, 1995, Ser. No. 08/566,414. 

FIG. 2 shows the major functional components that may 
be contained in one type of node which may be used in 
system 100, programmable switching node 1026. Digital or 
analog network/line interfaces 206 are terminated on a series 

40 of line card input/output (10) cards 204. A series of digital 
network Tl, El, Jl or analog trunk/line line cards 208 
communicate with line card IO cards 204 over line card (LC) 
IO lines 210. Line cards 208 are also interfaced with 
redundant switching buses 212a and 2126 (collectively and 

45 generally referred to as switching buses 212). Other types of 
network/line interfaces (and related line cards) such as DS3, 
SONET, SS7, ISDN or others may also be provided. 

Diverse communications services such tone detection and 
generation, conferencing, voice recorded announcements, 

50 call progress analysis, speech recognition, ADPCM com- 
pression and many others are provided by one or more 
multifunction digital signal processing (MFDSP) cards 214. 
Details of the architecture and operation of MFDSP cards 
214 and other optional cards, as well as buses 212, are 

55 disclosed in commonly owned U.S. Pat. No. 5,349,579. A 
ring (network) 10 card 216 serves as an interface between 
inter-nodal network 110 and a nodal switch 202 of the 
present invention. A host interface may be provided as noted 
above to establish a communication link with host 104. It 

60 should be understood that other cards may be added to or 
removed from the illustrative switch 1026. 

In contrast to switching node 1026, voice processing 
resource node 102c (FIG. 1) does not necessarily include 
line cards 208 and line card IO cards 204 since such a node 

65 need not interface with a PSTN or other network. However, 
such nodes may include additional components, such as a 
standard voice processing buses for communicating with, 
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for example, voice processing resources. For example, Dia- 
logic Corporation of New Jersey produces a family of voice 
processing resource boards or cards which plug directly into 
certain standard voice processing buses and may be used in 
diverse applications including voice mail, fax mail, interac- 
tive voice response and others. 

The detailed construction of a preferred embodiment of 
nodal switch 202 and ring 10 card 216 of the present 
invention is shown in FIG. 3. A central processing unit 
(CPU), with associated RAM/ROM, 302 is connected in 
communicating relationship with an address/data bus 304. 
CPU 302 is also connected in communicating relationship 
with an HDLC bus (part of switching buses 212) for 
communication with other cards within the node and may, 
depending upon the configuration of nodal switch 202, also 
be connected in communicating relationship with host 104. 
A data transmitter 306 and data receiver 308 are connected 
in communicating relationship with address/data buses 304 
and a packet handling circuit 312. 

A high speed data receiver 314 is physically interfaced 20 
with inter-nodal network 110 for receiving information in 
the form of optical signals from that ring. Receiver 314 is 
preferably implemented with a Hewlett-Packard Company 
HDMP-1014 receiver chip, which is an emitter coupled 
logic (ECL) device. Conversion circuit 316 is connected to 
receive the output signals of receiver 314 and produce 
output signals that are compatible with transistor-transistor 
logic (TTL). The output of conversion circuit 316 is applied, 
through a buffer 318, to the distributed synchronization 
system 326 of the present invention. The output of system 30 
326 is applied to packet handling circuit 312 that transfers 
data to/from data receiver 308 and data transmitter 306, 
respectively. A buffer 328, conversion circuit 330, and high 
speed data transmitter 332 perform functions which are 
complementary to those of buffer 318, conversion circuit 35 
316 and data receiver 314, respectively. Transmitter 332 is 
preferably implemented with a Hewlett-Packard Company 
HDMP-1012 transmitter chip. 

Receiver 314 includes circuitry which recovers a source 
node's clock signal from a received transmission and dis- *o 
tributes it as a recovered ring clock 322 to the components 
of the nodal switch 202 dedicated to receiving frame 
packets, including the distributed synchronization system 
326 of the present invention. A clock processor 334 lhat 
generates a local frame synchronization pulse 324 for use by 45 
the distributed synchronization system 326. Local frame 
synchronization pulse 324 is derived from a network refer- 
ence clock provided to the implementing node, typically 
from the PSTN or a private network. 

A ring oscillator 336 generates a local ring clock 320 used 
by the components of nodal switch 202, including distrib- 
uted synchronization system 326, to transmit frame packets. 
Further details of the structure and operation of nodal switch 
202 may be found in commonly owned U.S. Pat No. 
5,544,163. 

B. Distributed Synchronization System 
1. In General 

To ensure that the ring latency of the distributed network 
system 100 does not interfere with the transmission of 
synchronous (PCM) data, the present invention operates to 
delay the retransmission of a given frame packet until the 
occurrence of a predetermined frame synchronization signal 
(pulse), thereby dynamically adjusting the ring latency to an 
integer number of frame cycles. To avoid dataflow errors due 
to clock deviations between asynchronously- 65 
communicating nodes, the present invention performs the 
receive and retransmit functions asynchronously, maintain- 
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ing the data throughput at an optimal rale to prevent dataflow 
errors while avoiding excessive increases in ring latency. 

For the purpose of maintaining both word and frame 
synchronization between geographically-distributed nodes 
of an asynchronous network, the present invention com- 
prises two related, yet functionally distinct, subsystems: a 
frame synchronization subsystem operating in a designated 
master node; and a word resynchronization subsystem oper- 
ating in the remaining (non-master) nodes. Each subsystem 
works in conjunction with a unique frame packet discussed 
in detail below. 

Referring now to FIG. 4, the preferred embodiment of the 
distributed synchronization system 326 comprises a syn- 
chronization data FIFO 402, a word resynchronization sub- 
system 404 and a frame synchronization subsystem 406. 
Asynchronous data in the form of frame packets 502 (FIG. 
5A) arrive on data input line 401 and are written into data 
FIFO 402. Data which is read from data FIFO 402 appears 
on a data output line 403. 

Word resynchronization subsystem 404 monitors, via line 
412, frame packets presented to the input of data FIFO 402. 
Subsystem 404 also monitors the level of data in the data 
FIFO 402 and controls the writing of the presented frame 
packets into the data FIFO 402 via a control/status line 408. 
Subsystem 404 also receives as inputs recovered ring clock 
322 to control the writing of data into the data FIFO 402; and 
a local ring clock 320 to control the reading of data from data 
FIFO 402. 

Frame synchronization subsystem 406 also monitors the 
frame packets presented to the data FIFO 402. Subsystem 
406 also monitors the level of data in the data FIFO 402 and 
controls the reading and writing of frame packets into the 
data FIFO 402 via the control/status line 410. Since sub- 
system 406 also asynchronously performs the reception and 
retransmission of frame packets, it receives as inputs the 
recovered ring clock 322 and local ring clock 320, both of 
which are utilized for similar purposes as in the word 
resynchronization subsystem 404. In addition, subsystem 
406 receives the local frame synchronization pulse 324 that 
it uses to transmit the frame packets from FIFO 402. 

FIG. 5A shows a general structure for a frame packet 502 
for exchanging information over the inter-nodal network 
110. Each frame packet 502 comprises a number of fields, 
each containing one or more words of data, control infor- 
mation or fill frames (i.e., non-data). A frame synchroniza- 
tion field 504 provides an indication of the beginning of a 
frame packet 502. A payload field 506 comprises a number 
of subpackels, each containing data for transmission among 
nodes served by inter-nodal network 110. Payload field 506 
may contain any type of data within its subpackets, includ- 
ing circuit switched data, packet switched data, voice pro- 
cessing data and others. A dead zone field 508, which does 
not contain valid data and whose length or duration is 
dynamically-adjustable, is used for synchronization pur- 
poses as described below. Additional information regarding 
the structure of the subpackets as well as details of various 
packet structures for transferring different types of 
information, is described in commonly owned U.S. Pat No. 
5,544,163. 

FIG. 5B, in conjunction with FIG. 5A, illustrates a 
preferred approach for allocating the bandwidth of inter- 
nodal network 110 for the purpose of transferring data 
among nodes. Transfer of data over the network is preferably 
made within framing windows 510, each of which is 125 //s 
in duration. A period of 125 /is is preferred since it corre- 
sponds with the sampling rate (8 kHz) of most widely used 
circuit switched network protocols, meaning that the values 
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of circuit switched data may change every 125 f&. Thus, by 
requiring that all inter-nodal transfers of circuit switched 
data take place in less than 125 /zs, inter-nodal network 110 
ensures that all such data is transferred before any value 
changes. This also permits inter-nodal network 110 to oper- 
ate asynchronously with respect to the PSTN or private 
networks 106,118 (FIG. 1). 

2. Word Resynchronization Subsystem 

With reference now to FIGS. 6-8, the detailed operation 
of word resynchronization subsystem 404 is described. FIG. 
6 is a functional block diagram of the word resynchroniza- 
tion subsystem 404 and data FIFO 402 of the distributed 
synchronization system 326 of the present invention. The 
word resynchronization subsystem 404 generally includes a 
write controller 602, a read controller 604, an initializer 606 
and a counter 608 connected in communicating relationships 
by a control bus 626. Write controller 602 controls the 
writing of data into the data FIFO 402 via a write enable 
(WEN) signal line 610. Read controller 604 controls the 
reading of data from data FIFO 402 via a read enable (REN) 
signal line 612. 

Write controller 602 and read controller 604 control the 
flow of data through data FIFO 402 in response to a 
time-varying data level 620 of the FIFO as well as which 
field of a frame packet 624 is currently present at the input 
of data FIFO 402. More specifically, the controllers 602,604 
cooperate to maintain data level 620 between an upper 
threshold level 618 and a lower threshold level 622. Thresh- 
olds 618,622 are preferably chosen to define an optimal 
range of data level 620 to absorb phase differences between 
the clocks of the source node and the destination node 
without contributing excessively to ring latency. Upper 
threshold level 618 is preferably represented by a program- 
mable almost full (PAF) flag 614, while the lower threshold 
level 622 is preferably represented by a programmable 
almost empty (PAE) flag 616. Together, the two flags 
614,616 provide a current indication of the data level 620. 

PAE flag 616 indicates when data level 620 is below the 
associated lower threshold level 622. Similarly, PAF flag 
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FIFO 402 is cleared, when a node is initialized prior to 
commencing communications over the network and the like. 

At reset stale 702, write controller 602 initially sets the 
WEN control line 610 inactive, thus preventing the writing 
of data into data FIFO 402 until the necessary conditions are 
present. Upon completion of these initialization procedures, 
write controller 602 advances to write suspend state 704 as 
shown by state transition line 708. 

While write controller 602 is at write suspend state 704, 
it maintains WEN control line 610 inactive while monitoring 
PAF flag 614 and control bus 626. If PAF flag 614 is active, 
then data level 620 is above the upper threshold 618 as 
discussed above. Under such conditions, write controller 
602 continues to remain in the write suspend state 704 unless 
and until either a frame packet is received or data level 620 
falls below upper threshold 618. When data level 620 is 
below upper threshold 618 and PAF flag 614 is thus inactive, 
write controller 602 will allow data to be written into the 
data FIFO 402. Thus, when a frame packet is presented at the 
input of data FIFO 402 or when data FIFO 402 is not almost 
full, then write controller 602 advances to write stale 706 as 
shown by state transition line 710. 

At write state 706, write controller 602 sets WEN line 610 
active to enable writing of data into data FIFO 402. The 
writing of data continues until two conditions simulta- 
neously occur. If counter 608 (via control bus 626) indicates 
that dead zone field 508 is currently present at the input to 
data FIFO 402 which means that the payload field 506 has 
been completely written into the FIFO) and the data level 
620 is above the upper threshold 618 (and is thus above the 
desired optimal range), the writing of data is suspended. 
Thus, if payload field 506 of the current frame packet is 
completely written into FIFO 402 and the FIFO is almost 
full, write controller 602 advances to write suspend state 704 
as shown by state transition line 712. 

Referring now to FIGS. 6 and 8, read controller 604 
preferably operates in accordance with a state machine 800 
having three states: a reset state 802, a read suspend state 
804 and a read state 806. Initially, read controller 604 is at 


614 indicates when data level 620 is above the associated *o reset state 802, Read controller 604 may be reset for any 


upper threshold level 618. When data level 620 is at or 
below the lower threshold 622, PAE flag 616 is in its active 
state whereas when the data level 620 is at or above the 
upper threshold 618, PAF flag 614 is in its active slate. 
Alternatively, when data level 620 is above lower threshold 
622 and below upper threshold 618, the PAE and PAF flags 
arc both inactive. Thresholds 618 and 622 are initially set by 
initializer 606 via initialization control line 628 to predeter- 
mined levels. 


number of reasons such as those discussed above with 
respect to write controller reset stale 702. At reset state 802, 
read controller 604 sets REN control line 612 inactive to 
prevent the reading of data from FIFO 402 until the neces- 
45 sary conditions are met. Upon completion of the reset/ 
initialization procedures, read controller 604 advances to 
read suspend state 804 as shown by state transition line 808. 

While read controller 604 is at read suspend state 804, it 
maintains REN control line 812 inactive while monitoring 


As noted above, frame packet 624 comprises a number of 50 PAE flag 616 and control bus 626. If PAE flag 616 is active, 


fields. The operations performed by word resynchronization 
subsystem 404 depend, in part, upon which field of the frame 
packet 624 is present at the input to data FIFO 402. This is 
determined by counter 608 via monitor line 412. Upon 
receipt of a frame synchronization field counter 608 begins 
to count the number of words received by data FIFO 402 and 
resets an internal timer. When counter 608 reaches a prede- 
termined value corresponding lo the beginning of the dead 
zone field 508, the counter transmits a signal on control bus 
626 instructing write controller 602 and read controller 604 
to resynchronize. 

Referring now to FIGS. 6 and 7, write controller 602 
preferably operates in accordance with a state machine 700 
having three states: a reset state 702, a write suspend state 
704 and a write stale 706. Initially, write controller 602 is at 
reset state 702. Write controller 602 may be reset for any 
number of reasons, such as upon receipt of power, when data 
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then data level 620 is below lower threshold 622 as dis- 
cussed above. Under such conditions, the read controller 604 
continues to remain in read suspend state 804 unless a frame 
packet is received or data level 620 rises above lower 
threshold 622. When either of those events occurs, read 
controller 604 advances to read state 806. 

At read slate 806, read controller 604 sets REN control 
signal 612 active to enable reading of data from FIFO 402. 
So long as counter 608 indicates, again, via control bus 626, 
that dead zooe field 508 is currently present at the input to 
the data FIFO 402 (i.e., the payload field 506 has been 
completely written into the FIFO) or data level 620 is in the 
optimal range, reading of data remains enabled and data will 
continue to be read from data FIFO 402. 

However, if counter 608 indicates that dead zone field 508 
is currently present at Ihe input to data FIFO 402 and data 
level 620 is simultaneously below lower threshold 622 (and 
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is thus below ihe optimal range), the reading of data is 
suspended. Thus, if pay load field 506 of the currently- 
presented frame packet is completely written into FIFO 402 
and the FIFO 402 has become almost empty, read controller 
602 advances to read suspend state 804 as shown by state 5 
transition line 812. 

Write and read controllers 602,604 intemperate to main- 
tain data level 620 within between upper threshold 618 and 
lower threshold 622. As noted above, write and read con- 
trollers 602,604 perform their functions in response to an 10 
internally-generated synchronization instruction based upon 
the slates of PAF and PAE flags 614,616 and the portion of 
frame packet 502 present at the input of data FIFO 402. 

Upon receipt of a frame synchronization field counter 608 
begins counting the data words which follow that field (i.e., is 
the words in payload field 506). When counter 608 reaches 
a predetermined number of words representing the end of 
payload field 506 (and the beginning of dead zone field 508), 
counter 608 issues a resynchronization signal over control 
bus 626 which causes write and read controllers 602,604 to 20 
check the status of the PAE and PAF flags. If another frame 
packet is not subsequently received, counter 608 continues 
to reset and increment, each time generating a resynchroni- 
zation instruction which causes word resynchronization sub- 
system 404 to resynchronize. In other words, counter 608 25 
begins incrementing when a frame synchronization field 504 
is received and, upon reaching its maximum predetermined 
value, counter 608 resets and begins again in anticipation of 
receiving another frame synchronization field 504. Thus, 
resynchronization is guaranteed to occur regardless of 30 
whether a frame synchronization field 504 is actually 
received. 

If, during resynchronization, write and read controllers 
602,604 determine that data level 620 is between upper and 
lower thresholds 618,622, then controllers 602,604 will 35 
allow continuous reading and writing of data through the 
data FIFO. However, if data level 620 is either above upper 
threshold 618 or below the lower threshold 622, then read 
and write controllers 602,604 will momentarily suspend 
writing or reading of data as needed to restore data level 620 40 
to within the optimal range. 

Upon initialization or invocation of a reset condition, 
write and read controllers 402,404 are placed in reset states 
702,802, respectively. While the controllers are in their 
respective reset states, initializer 606 sets the values of upper 45 
threshold 618 and lower threshold 622 at respective prede- 
termined values to define an optimal range for data level 
620. By setting those thresholds, a desired optimal range for 
data level 620 is established prior to the presentation of valid 
data at the input of data FIFO 402. so 

It is important that data level 620 is built-up to within the 
established optimal range before valid data is received to 
avoid the immediate slippage of words (i.e., during the time 
delay that would be incurred in filling a completely empty 
FIFO to at least the almost empty level). During periods 55 
when no nodes are transmitting onto the network, the master 
node may generate fill frames or frame packets having 
payload fields 506 that contain no valid data. Such fill 
frames may be received and stored by all the nodes in the 
network implementing the word resynchronization sub- 60 
system 404 of the present invention. Thus, although no valid 
data is received, the fill frames are nonetheless stored in the 
data FIFO 402, thereby maintaining data level 620 in the 
optima] range prior to the receipt of valid data (i.e., prior to 
normal communications). 65 

Preferably, an optimal range for data level 620 is chosen 
based upon the expected deviations between the source and 
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destination node clocks in the network. In a preferred 
embodiment of the present invention, each node has a local 
oscillator which operates at 131.072 MHz±25 ppm (parts per 
million). The local clock of each node is preferably config- 
ured to operate at half the oscillator rate. Thus, the tolerance 
of each node's local clock is (131.072+2x25) or 1638 
words/second. That is, variations in the frequency of each 
node's local clock cause corresponding variations in data 
transmission by as much as 1638 words/second. The largest 
difference between two nodes on the network will occur 
when a node containing a local oscillator running at 
131.072+25 ppm is communicating with another node con- 
taining a local oscillator running at 131.072-25 ppm. This 
worst case scenario will result in an error condition of 
(2x1638) or 3276 words/second. In that scenario, a slip of 
one word can be expected every 305.25 /is. 

Note, however, that in addition to instructing the control- 
lers to resynchronize upon receipt of the dead zone, counter 
608 also instructs the write and read controllers to resyn- 
chronize when no frame synchronization field 504 is 
received at all. That is, counter 608 continually resets and 
increments (up to the expiration of its internal 125 fts timer) 
regardless of whether a frame synchronization field 504 
appears at the input of the data FIFO 402. This periodic 
resynchronization continues indefinitely and is interrupted 
only when a frame synchronization field 504 is received. 
Thus, the longest period of time between resynchronizations 
is when a frame synchronization field 504 is received 
immediately prior to the timer expiring. For example, if the 
packet frame contains 8125 words and lasts 125 ps and dead 
zone field 508 is 100 words in length, then the timer will 
indicate the beginning of the dead zone at (8125-100 
words)xl5 nanoseconds) or 120.4 //s, with the longest 
duration between resynchronizations being (120.4x2) or 
240.8 /is. 

Because word resynchronization is preferably performed 
every frame, or once every 125 /rs, which is less than the 
time of an expected word slip of 305 fts, no word slips 
should occur. As a result, an optimal data FIFO level 620 
may be only a few words. In a preferred embodiment, upper 
threshold 618 and lower threshold 622 are determined based 
upon how often data FIFO 402 will be serviced by the 
controllers, the anticipated deviations between the source 
and destination clocks, the hysteric behavior of the 
subsystem, as well as the data FIFO's sensitivity to under- 
flow and overflow conditions. However, as one skilled in the 
relevant art would find apparent, other criteria may also be 
considered. In a preferred embodiment, the capacity of data 
FIFO 402 is significantly larger than upper threshold 618, to 
provide protection against overflow conditions. However, if 
the data FIFO level becomes too large, there will be sig- 
nificant delay through the node. Although it is desirable to 
maintain the data level 620 as low as possible to reduce the 
delay through the node, to provide protection against under- 
flow conditions there must be a data level sufficient to 
prevent loss of data in light of the above factors. These 
concerns are balanced against ring latency requirements to 
obtain an optimal range for data level 620. In a preferred 
embodiment, the optimal data level 620 is set at an 8 word 
depth, with the associated upper and lower thresholds 618, 
620 set at 8 and 12 words, respectively. A data FIFO level 
of 8 words will not cause significant delay through the node 
(8x15 ns-120 ns), while providing a conservative number of 
words to prevent slippage (despite the fact that none should 
occur given the above calculations). 

Note that writing of data is suspended only during the 
receipt of the dead zone field 508 when only fill frames 
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(non-valid data) ire received. As a result, some, all, or none 
of the dead zone field 508 will be stored in the data FIFO 
402. The portion that is stored is the amount necessary to 
guarantee that the frame packet which is subsequently read 
from the data FIFO 402 and transmitted at the rate of the 
local clock will average 125 /xs in length. Furthermore, the 
clock signal transmitted to another node will not contain 
jitter or wander components of the recovered source node's 
clock. 

3. Frame Synchronization Subsystem 

With reference now to FIGS. 9-11, the operation of frame 
synchronization subsystem 406 is described. FIG. 9 is a 
functional block diagram of subsystem 406 and data FIFO 
402 of the distributed synchronization system 326 of the 
present invention. Frame synchronization subsystem 406 
includes a write controller 902, a read controller 904, an 
initializer 906, and a counter 908. Write controller 902 
controls writing of data into data FIFO 402 via a write 
enable (WEN) signal line 910. Read controller 904 controls 
reading of data from data FIFO 402 via a read enable (REN) 
signal line 912. When the REN and WEN control lines are 
active, the data FIFO 402 is enabled to read and write data, 
respectively. 

A lower data threshold 922 is represented by a synchro- 
nization programmable almost empty (SYNCPAE) flag 916. 
When data level 920 is at or below lower threshold 922, 
SYNCPAE flag 916 is in its active state. Conversely, when 
data level 920 is above lower threshold 922, the SYNCPAE 
flag is in its inactive state. Lower threshold 922 is initially 
set by an initializer 906 via initialization control line 928 to 
a predetermined level (discussed below). 

Frame synchronization subsystem 406 generates an ini- 
tialization frame wheo the network is first initialized. Each 
non-master node receives and retransmits the initialization 
frame, causing each node to successively initialize its nodal 
switch. When the initialization frame returns to the master 
node, the master node itself then initializes with the knowl- 
edge that all other nodes in the network are ready to 
commence inter-nodal communications. The master node 
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while data FIFO 402 is filled with data, thereby automati- 
cally raising the data level by the amount that it was 
previously depleted. 

Referring to FIGS. 9 and 10, write controller state 
machine 1000 has three states: a reset state 1002, a write 
suspend state 1004, and a write state 1006. Initially, write 
controller 902 is at reset state 1002. 

Write controller 902 may be reset for any number of 
reasons, such as when initializer 906 sets the lower threshold 
level 922, upon the initial application or an interruption of 
power to the frame synchronization subsystem 406, and the 
like. 

At reset state 1002, write controller 902 sets the WEN 
control line 910 inactive to prevent the writing of data into 
data FIFO 402 until the necessary conditions are met. Upon 
completion of the initialization procedures, write controller 
902 advances to write suspend state 1004 as shown by state 
transition line 1012. 

While write controller 902 is at write suspend state 1004, 
it maintains WEN control line 910 inactive to prevent writes 
from occurring. During this time, counter 908 monitors the 
input of the data FIFO 402 to determine which portion of 
frame packet 901 is currently present. When counter 908 
detects a frame synchronization field 903 or does not detect 
a frame packet during a frame transmission, counter 908 
invokes a self-synchronization operation, instructing write 
controller 902 to begin writing data into the data FIFO 402. 
When this occurs, write controller 902 transitions from write 
suspend state 1004 to write state 1006 as shown by state 
transition line 1014, 

At write state 1006, write controller 902 sets WEN control 
line 910 active to begin writing the contents of any received 
frame packets into data FIFO 402. The writing of the frame 
packet fields continues until counter 908 indicates that dead 
zone field 907 is currently present at the input to the data 
FIFO 402. On the other hand, if counter 908 did not detect 
a frame packet during a frame transmission, then write 
controller 902 will write fill frames into data FIFO 402 while 
in write state 1006. In this circumstance, the counter 908 will 


then transmits a frame synchronization field designating the aq still indicate the point at which a dead zone 907 would 


frame boundaries around the ring. 

The determination of which field of frame packet 901 is 
present at the FIFO input is made by a counter 908 via a 
monitor line 412. When counter 908 detects a frame syn- 
chronization field 903, it counts up to the dead zone field 907 
then issues a self-synchronization command to write con- 
troller 902 to begin or suspend wrile operations. When 
counter 908 does not detect a frame synchronization field 
903 (and thus a dead zone field 907 as well) during a frame 
transmission, the counter issues a self-synchronization com- 
mand to write controller 902. Based upon the frame packet 
field and the invocation of self -synchronization, counter 908 
generates a signal on control bus 926 instructing write 
controller 902 to write or not write the received fields into 
data FIFO 402. 

Frame synchronization subsystem 406 dynamically 
adjusts the amount of data stored in data FIFO 402 to 
accommodate variations in ring latency. If, for example, the 
ring latency is increased suddenly due to a node failure and 
subsequent loop-back operations, then data level 920 in data 
FIFO 402 will either rise or fall depending upon the lime of 
the failure in relation to the frame that is being processed by 
the FIFO. However, frame synchronization subsystem 406 
automatically recovers because the next frame synchroniza- 
tion field 903 that appears at the output of data FIFO 402 
does so significantly before the occurrence of the local frame 
synchronization pulse 324. As a result, reads are suspended 
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normally have appeared at the input of data FIFO 402. 

[n other words, regardless of whether a frame packet or 
fill frames are being written into data FIFO 402, write 
controller 902 will transition to write suspend state 1004 
during a portion of the frame transmission. Thus, when dead 
zone field 907 of the current frame packet 901 is detected at 
the input to data FIFO 402 or when self -synchronization 
occurs, write controller 902 advances to write suspend state 
1004 as shown by state transition line 1010. As a result, 
write controller 902 writes only the frame synchronization 
and payload fields (and not the dead zone field) into data 
FIFO 402. If no frame packet is presented to data FIFO 402, 
then write controller 902 periodically transitions between 
write state 1004 and write suspend state 1006. 

Referring to FIGS. 9 and 11, read controller state machine 
1100 has three states: a reset state 1102, a read state 1104; 
and a read suspend state 1106. Initially, read controller 904 
is at reset state 1102. At the reset state 1102, read controller 
904 sets the read enable REN control line 912 inactive to 
prevent the reading of data from data FIFO 402 until the 
necessary conditions are present. Upon completion of the 
initialization procedures, read controller 904 advances to 
read stale 1104 as shown by state transition line 1112. 

While read controller 904 is at read state 1104, it sets the 
REN control line 912 active while it monitors the output of 
data FIFO 402. This will maintain data FIFO 402 at an 
optimal minimal level, somewhere close to empty, until a 
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frame packet has been received and stored in the FIFO. 
When the controller 904 determines that frame synchroni- 
zation field 903 is about to be read from the data FIFO 402, 
read controller 904 advances to read suspend state 1106 as 
shown by state transition line 1114. 

In read suspend state 1106, read controller 904 sets the 
REN control line 912 inactive to stop the reads from tile data 
FIFO 402 from continuing. Read controller 904 then waits 
until the simultaneous occurrence of two conditions: the 
receipt of the local frame synchronization pulse 324 and the 
accumulation of data in FIFO 402 such that data level 920 
is above the lower threshold 922. If the local frame syn- 
chronization pulse 324 occurs and the data FIFO 402 is not 
almost empty, then there is a sufficient amount of data in the 
data FIFO 402 to safely read data without causing underflow 
conditions. When data level 920 is below lower threshold 
922, read controller 904 remains in the read suspend state 
1106. On the other hand, when the data level 920 is above 
the lower threshold 922, the read controller 904 will allow 
data to be read from data FIFO 402 upon receipt of a local 


Since the write controller 902 and the read controller 904 
operate asynchronously, data level 920 in the data FIFO 402 
may drift upward. To avoid having an unnecessary number 
of fill frames in data FIFO 402 when a frame packet 

5 containing valid data is received, write controller 902 pref- 
erably periodically ceases writes operations while the read 
controller 904 continues read operations. That is, if a frame 
synchronization field 903 is not detected while the write 
controller 902 is in the write state 1006 and the counter 908 

10 is incrementing, then write controller 902 transitions to the 
write suspend state 1004, allowing read controller 904 to 
operate to reduce data level 920 during the time that dead 
zone field 907 is presented to data FIFO 402. Write con- 
troller 902 will preferably transition between those two 

15 states indefinitely until a frame synchronization field 903 is 
received. 

Upon receipt of a frame synchronization field 903, 
counter 908 will reset and begin to count the number of 
words written into data FIFO 402. Write controller 902 will 


frame synchronization pulse 324. Thus, when a frame packet 20 either remain in write state 1006 or transition to write state 


911 is presented at the output of data FIFO 402 and data 
level 920 is not almost empty, then read controller 904 
advances to read state 1104 as shown by state transition line 

mo. 

If a local frame synchronization pulse 324 occurs prior to 25 
the time the data level 920 exceeds lower threshold 922, read 
controller 904 will remain in read suspend state 1106 and 
continue to accumulate data until the next occurrence of the 
local frame synchronization pulse 324. Thus, when a the 
local frame synchronization pulse 324 occurs and data level 
920 is above lower threshold 922, read controller 904 
transitions to read state 1104, releasing frame synchroniza- 
tion field 903 and payload field 905. When the next frame 
synchronization field appears at the output of data FIFO 402, 
read controller 904 will return to the read suspend state 1106 
until the next local frame synchronization pulse 324 occurs. 

Write and read controllers 902,904 perform their func- 
tions asychronously with respect to each other. However, 
their functions are coordinated to ensure that a frame packet 
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1006 from the write suspend state 1004. When counter 908 
reaches a predetermined value indicating that frame syn- 
chronization field 903 and payload field 905 have been 
written into data FIFO 402 and dead zone field 907 is 
currently present at the input of data FIFO 402, then counter 
908 instructs write controller 902 to cease writing into data 
FIFO 402 (since the received dead zone field 907 is not 
retransmited by the subsystem 406). 

When frame synchronization field 903 appears at the 
output of data FIFO 402, read controller 904 advances to 
read suspend state 1106, causing data FIFO 402 to start 
filling up. Upon the occurrence of a local frame synchroni- 
zation pulse 324, read controller 904 reads the frame syn- 
chronization field 903 and the payload field 905 from data 
FIFO 402. Read controller 904 continues to read data from 
the FIFO until it detects a new frame synchronization field 
903 presented at the output of the data FIFO 402. When this 
occurs, the read controller ceases to read from the FIFO, 
causing fill frames to be added to the frame packet until it 


is released from the implementing master node 102a at such 40 contains a sufficient number of words to precisely equal a 


time that the ring latency will be an integer multiple of frame 
packets. Write and read controllers 902, 904 maintain an 
optima] amount of frame packet fields in data FIFO 402 so 
that a frame packet is ready for transmission upon the 
occurrence of a pulse while not causing excessive delay 
through the master node or exposing the node to potential 
underflow conditions. This coordinated operation of the 
write and read controllers is discussed below. 

When write and read controllers 902,904 are in their 
respective reset states, WEN and REN control lines 910, 912 
are set inactive to prevent any data from being stored in, or 
retrieved from, data FIFO 402. After initialization, write 
controller 902 advances to write suspend stale 1004 and read 
controller 904 advances to read state 1104. No data is written 
into the data FIFO 402 until either a frame synchronization 
field 903 is detected at the input of data FIFO 402 or a 
self-synchronization signal is received. However, data is 
read from data FIFO 402, although initially no valid data 
will be presented at the output of the FIFO. 

If a frame synchronization field 903 is not detected within 
a predetermined time equivalent to a frame transmission 
(i.e., 125 fts). then counter 908 generates a self- 
synchronization signal. This causes write controller 902 to 
advance to write state 1006, while read controller 904 
remains in read state 1104. Thus, any data received will be 
written into and read from the data FIFO 402 with little or 
no delay since the data FIFO is essentially empty. 
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frame transmission. Thus, a new dead zone 909 is effectively 
created by the ceasing of read operations. This guarantees 
that each frame packet 911 transmitted from the master node 
102a contains an exact predetermined number of words for 
a frame transmission. 

Since write controller 902 does not write (i.e., discards) 
the dead zone field 907 while read controller 904 is per- 
forming read operations, data FIFO 402 will be depleted by 
an amount of words approximately equivalent to the size of 
the dead zone field 907. Likewise, when read controller 904 
is in read suspend state 1106 while write controller 902 is in 
the write state 1006, data level 920 will increase by the size 
of the dead zone field 907. 

Thus, to avoid dataflow errors from occurring, data FIFO 
402 must be maintained with a number of words equal to at 
least the number of words contained in dead zone field 907. 

An additional requirement is that data FIFO 402 must 
contain a quantity of data, referred to as the "remainder", 
that is equivalent to the difference between the ring latency 
and an integer multiple of the frame. If the remainder is less 
than the size of dead zone field 907, then the remainder plus 
a complete frame of data (payload field 905 and frame 
synchronization field 903) is stored in the FIFO since the 
data level 920 will be below the lower threshold 922 when 
a pulse 324 occurs, causing the read controller 904 to stay 
in the read suspend state 1106 until a following pulse 324 
after another frame of data has been written into the FIFO. 
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On the other hand, if the remainder is greater than the size 
of dead zone field 907, then only the remainder will be 
stored in the data FIFO since the remainder number of words 
in the FIFO 402 will always be above the lower threshold 
922 set at the size of the dead zone field 907. As a result, 
SYNCPAE flag 916 will never become active once a frame 
packet has been received. 

In a preferred embodiment of the present invention, the 
distributed synchronization system 326 comprises both a 
frame synchronization subsystem 406 implemented operat- 
ing in a master node and a word resynchronization sub- 
system 404 implemented in each non-master node. 
However, as one skilled in the relevant art would find 
apparent, distributed synchronization system 326 may be 
implemented with either the frame synchronization sub- 
system 406 or the word resynchronization subsystem 404 
individually. 

It is noted, however, that a network implementing both 
subsystems enables continued, robust synchronized commu- 
nications to occur between asynchronous nodes in the net- 20 
work while preventing dataflow errors and without contrib- 
uting excessively to network latency. 

As a result, in a preferred embodiment of the present 
invention, each network node is implemented with both 
subsystems. Accordingly, the size of FIFO 402 is preferably 25 
large enough to accommodate both of these implementa- 
tions. Thus, FIFO 402 preferably has a byte capacity which 
is slightly larger than the amount of data contained within 
one frame. This will provide an implementing node with the 
capacity of acting as a master node and performing frame 30 
synchronization. Alternatively, the node may be a non- 
master node, performing solely word resynchronization. 
Furthermore, if such a node is implemented in a network 
system that is not distributed, frame synchronization is not 
required, thereby enabling the FIFO 402 to be considerably 
smaller. 

While the invention has been particularly shown and 
described with reference to preferred embodiments thereof, 
it will be understood by those skilled in the art that various 
changes in form and details may be made therein without ^ 
departing from the spirit and scope of the invention. 
Furthermore, the terms and expressions which have been 
employed are used as terms of description and not of 
limitation, and there is no intention, in the use of such terms 
and expressions, of excluding any equivalents of the features 45 
shown and described or portions thereof, but it is recognized 
that various modifications are possible within the scope of 
the invention claimed. 

What is claimed is: 

1. A distributed synchronization system for use in a node 
of an expandable telecommunications system including a 
plurality of nodes interconnected by an inter-nodal network, 
with one of said nodes being a master node, the system, 
comprising: 

a synchronization data memory implemented in a master 
node and each of said nodes which are non-master 
nodes for storing predetermined portions of an inbound 
frame packet received from a source node and from 
which information is retransmitted to a destination 
node on the network; 

a frame synchronization subsystem implemented in said 
master node, and connected in communicating relation- 
ship with said memory, configured to release an out- 
bound frame packet at the beginning of an 
independently-determined frame cycle, based upon the 
occurrence of a local frame synchronization pulse, and 
wherein said frame packets include a frame synchro- 
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nization field indicating the beginning of a new frame 
packet, a payload field containing valid data and a 
dynamically-adjustable dead zone comprising a non- 
valid data field; and 
a word resynchronization subsystem implemented in said 
master node and each non-master node and connected 
in communicating relationship with said memory and 
configured to control storage of said predetermined 
portions using a recovered source node clock signal and 
to control retransmission of said information according 
to a local clock signal, such that each non-master node 
receives a frame packet at predetermined interval of 
time whereby dataflow errors due to phase differences 
in source and destination nodal clocks are substantially 
avoided. 

2. The synchronization system of claim 1 wherein the 
frame synchronization subsystem comprises: 

a counter for generating a signal indicative of which 
portion of the inbound frame packet is present at an 
input to said memory; 

an initializer connected to said memory for establishing a 
threshold data level of said memory; 

a read controller connected to said memory and respon- 
sive to a signal indicative of whether a current data 
level of said memory is greater than said threshold data 
level to alternately enable or suspend reading of infor- 
mation from said memory, a master clock signal for 
reading information from said memory for inclusion in 
said outbound frame packet, and a frame synchroniza- 
tion pulse which is derived from a network reference 
clock and is used to trigger the release of said outbound 
frame packet; 

a write controller connected to said memory and respon- 
sive to said recovered source node clock signal to 
alternately enable or suspend writing of information 
into said memory; 

said counter, initializer and read and write controllers 
connected in communicating relationships by a control 
bus. 

3. The synchronization system of claim 1 wherein the 
word resynchronization subsystem further comprises: 

a counter for counting a number of words that have been 
written into the data memory following receipt of said 
frame synchronization field and resetting a timer, for 
generating a signal indicative of which portion of the 
inbound frame packet is present at an input to said 
memory, and to instruct a read controller and a write 
controller to resynchronize when the counter reaches a 
predetermined value corresponding to the presence of 
said no d- valid data field at said memory input; 

an initializer connected to said memory for establishing 
upper and lower threshold data levels of said memory; 

said read controller connected to said memory and 
responsive to a signal indicative of whether a current 
data level of said memory is greater than said lower 
threshold data level, and a master clock signal for 
controlling the reading of information from said 
memory; 

said write controller connected to said memory and 
responsive to a signal indicative of whether a current 
data level of said memory is less than said upper 
threshold level, and a recovered source node clock 
signal for controlling the writing of information into 
said memory; 

wherein said read and write controllers function in coop- 
erating relationship to maintain the data level of said 
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memory within an optimal range between said upper 
and lower threshold levels; 
said counter, initializer and read and write controllers are 
connected in communicating relationships by a control 
bus. 

4. The synchronization system of claim 3 wherein said 
read and write controllers function to substantially compen- 
sate for phase differences between said recovered source 
node clock signal and said local clock signal by receiving 
and storing data in accordance with said recovered source 
node clock signal and retrieving and retransmitting the data 
in accordance with said local clock signal. 

5. A telecommunications switch which is operable as a 
node in an expandable telecommunications system, said 
switch comprising: 

one or more nodal switches for dynamically connecting or 
disconnecting communication paths with respect to 
various ones of a plurality of ports and transmitting and 
receiving packetized information over an inter-nodal 
network, said inter-nodal network for providing com- 
munications between said telecommunications switch 
and other nodes associated with said system; and 

a word resynchronization subsystem including a memory 
and a controller, wherein said memory is implemented 
in a master node and each non- master node for storing 
predetermined portions of an inbound frame packet 
received from a source node and from which informa- 
tion for inclusion in an outbound frame packet is 
retrieved, and said controller for regulating a flow of 
information through said memory in response to a 
time- varying data level of the memory, and maintaining 
said data level within a predetermined optimal range; 
and 

frame synchronization subsystem implemented in said 
master node, and connected in communicating relation- 
ship with said memory, configured to release an out- 
bound frame packet at the beginning of an 
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independently-determined frame cycle, based upon the 
occurrence of a local frame synchronization pulse, and 
wherein said frame packets include a frame synchro- 
nization field indicating the beginning of a new frame 
packet, a payload field containing valid data and a 
dynamically -adjustable dead zone comprising a non- 
valid data field. 
6. A method of synchronizing data in a node of an 
expandable telecommunications system, the system com- 
prising a plurality of switching nodes interconnected by an 
inter-nodal network withone of said nodes being a master 
node, said method comprising the steps of: 

(a) at the master node, transmitting a first frame packet at 
the beginning of an independently-determined frame 
cycle: 

(b) at each frame packet, a frame synchronization field 
indicating the beginning of said frame packet, a pay- 
load field including a number of subpackets, each 
containing data for transmission among nodes served 
by the inter-nodal network, and a dynamically- 
adjustable dead zone for synchronization purposes, 
comprising a non-valid data field; 

(c) at a first non-master node, storing in a memory 
predetermined portions of said frame packet, and 
retrieving from said memory information which is to be 
retransmitted to another non-master node or the master 
node on the network, and controlling said retransmis- 
sion of said information using a local frame synchro- 
nization pulse, such that each non-master node receives 
a frame packet at predetermined interval of time 
whereby data flow errors due to phase differences in 
source and destination nodal clocks are substantially 
avoided; and 

(d) repeating step (c) at each non-master node until said 
first frame packet returns to said master node. 
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ABSTRACT 


A communication system including a collection of modules 
coupled in a ring architecture which integrates synchronous 
and asynchronous message transmission. Asynchronous 
data packets and synchronous voice packets are exchange on 
a single communication link. Packetized information 
exchange references a fixed length window with all syn- 
chronous data being exchanged at the beginning of each 
window and with asynchronous data exchanged during the 
remaining portion of each window. Virtual circuits utilizing 
the synchronous packets can deliver telephone conversa- 
tions without degradation in voice quality and yet the system 
can also transmit asynchronous data packets. 

2 Claims, 5 Drawing Sheets 
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COMBINED SYNCHRONOUS AND 
ASYNCHRONOUS MESSAGE 
TRANSMISSION 

RELATED APPLICATION 

The present invention is a continuation-in-part of appli- 
cation Ser. No. 60/098,297 filed Aug. 27, 1998 entitled 
"Combined Synchronous and Asynchronous Message 
Transmission. " 

BACKGROUND OF THE INVENTION 
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Packet switching systems transmit data by breaking the 
data into relatively small manageable pieces called packets. 
Packet switching can be used to transmit data in both 
computer networks and in telephone voice networks. Tele- 
phone packet switching networks transmit a series of pack- 
ets over the same route in the network. Such systems in 
effect establish a virtual circuit from the point where a series 
of packets enters the network to the point where the packets 
are delivered. Packet switching networks establish virtual 
circuits through the network in order to transmit voice 
without delay and distortion. 

Protocols such as the Internet ITPC protocol can transmit 
voice without establishing a virtual circuit connection, 
however, voice transmission using this type of protocol 
generally has less quality than voice transmitted using 
protocols which establish virtual circuits between the input 
point and the output point in the network. 

Today, some voice transmissions are being made over 
packet protocols (such as the Internet) which do not estab- 
lish virtual circuits. Voice connections over such circuits are 
of relatively low quality. The packet protocols which arc 
used in the public telephone network are packet protocols 
which establish virtual circuits and which transmit all the 35 
packets that constitute a conversation over the same route 
through the network. Thus they provide high quality con- 
nections. 

Data communication protocols can be characterized as 
either synchronous or asynchronous. Examples of widely 
used synchronous protocols are the X.25 protocol, and the 
frame relay protocol. Examples of widely used asynchro- 
nous protocols arc the Ethernet, FDDI and ATM protocols. 
The X.25 protocol, the frame relay protocol and the ATM 
protocol arc widely used in telephone systems. The Ethernet 
protocol and the FDDI ring protocol are widely used in local 
area networks (LANS) and wide area networks (WANS) that 
are used to interconnect computer systems. 

There are various well known techniques for controlling 
asynchronous networks. One technique termed "carrier 
sense, multiple access with collision detection (CSMA/CD) 
is used in Ethernet networks. Another technique called token 
passing is used in FDDI ring networks. 

Explanations of various synchronous and asynchronous 
protocols, and an explanation of CSMA/CD and FDDI ring 
networks is for example given in a book entitled "Voice and 
Data Communications Handbook" by Regis J. Bates and 
Donald Gregory which is published by McGraw Hill. 

SUMMARY OF THE INVENTION 60 

The present invention provides a ring protocol and system 
that combines synchronous and asynchronous transmission 
techniques. The ring can interconnect a number of modules 
and be utilized to transmit both fixed and variable packets 65 
between the modules. Communication time is broken into a 
sequence of fixed length windows. At the beginning of each 
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window the modules communicate using a synchronous 
protocol. That is, at the beginning of each window, if any 
unit has synchronous traffic, such traffic is transmitted using 
a synchronous ring protocol and fixed length packets. Virtual 
circuits can be established between the modules using the 
synchronous fixed length packets communicated at the 
beginning of each window. When it is desired to establish a 
virtual circuit between any of the modules in the ring, each 
module is assured that at the beginning of each window, 
space will be allocated to transmit a synchronous fixed 
length packet to another module in the ring. The windows 
occur frequently enough that a virtual voice grade circuit can 
be established between the modules. After all synchronous 
packets required during any window have been transmitted, 
asynchronous variable length data packets are transmitted 
around the ring. Limits are provided relative to the number 
of asynchronous packets any one module can transmit, 
thereby avoiding monopolization of the ring by any one 
module. The modules are synchronized by a periodically 
circulating a timing control character around the ring. 

BRTEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is an overall block diagram of the preferred 
embodiment. 

FIG. 2 is a diagram of the package assembler and disas- 
sembler (PAD) portion of the modules for coupling to analog 
telephone devices. 

FIG. 3 shows a repeating sequence of fixed-length time 
windows used in allocating data traffic between the modules 
of FIG. 1. 

FIG. 4 is a system-level flow chart illustrating overall 
operation of the communication system of FIG. 1. 

FIG. 5 is a program flow chart showing the operation of 
the module which executes certain control among the mod- 
ules of FIG. 1. 

FIG. 6 is a flow chart showing the operation of modules 
of FIG. 1 when receiving and processing information 
exchanged there-betweeo. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

A preferred embodiment of the invention is shown in FIG. 
1 and described herein. FIG. 1 shows a communication 
system 8 which supports integrated exchange of asynchro- 
nous and synchronous data between a number of modules 
10a, lOfc, etc. The data exchanged between the modules 
includes data traffic from and to computers 18 and voice 
traffic from conventional telephone devices 20 and 22. The 
data traffic from and to the computers 18 can include all of 
the various types of data traffic conventionally generated by 
computers such as video data, pc -phone data, etc. The 
system shown in FIG. 1 establishes "virtual circuits" 
between modules 10 for telephone traffic (i.e. synchronous 
transfer) and to also manages exchange of asynchronous 
information transfer between modules 10, 

As shown in FIG. 1 system 8 includes a collection of 
modules 10 organized in a ring architecture. Information 
travels from one module 10 to a successive module 10 as 
required. The information on the ring can be divided into 
three categories, namely, data packets, tokens, and control 
characters (including a liming character). As used herein the 
following terms have the following meaning. 

A"byte" consists of ten bits. Eight data bits are coded into 
a ten bit byte. Since a ten bit byte is used to encode 8 bits 
of data, a byte can be decoded into 256 different data words 
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plus 768 additional decodes. Some of the additional decodes 
are used to form control characters, a timing character, and 
tokens. Other ones of the additional decodes are used for 
purposes unrelated to the present invention. Such decoding 
is conventional. 

A "control character" consists one byte. The specific byte 
that forms each control character is selected from the 
decodes which do not form data words. There are three 
control characters which are used to implement the present 
invention. One control character indicates the start of a 
window, a second indicates the end of synchronous data 
transfer, and the third indicates the end of a window. When 
a module receives a control character it immediately retrans- 
mits the character to the next module in the ring. 

A "token" consists of two bytes of data. As with control 
characters, the specific bytes that form each token arc 
selected from the decodes which are not otherwise assigned. 
When a module receives a token, it only retransmits the 
token to the next module if certain conditions have been met. 
There are two tokens used in the implementation of the 20 
present invention. One token indicates that a module should 
begin transmitting its synchronous data and the second 
indicates that a module should begin transmitting asynchro- 
nous data. A module only passes a token to the next module 
after a module that receives a token has completed the task 25 
initialed by the token. 

A "timing character" consists of one byte. This one byte 
is selected from the decodes not otherwise assigned. 

A "synchronous packet" consists of 16 bytes of data plus 
three bytes of address information. 

A "asynchronous packet" consists of 64 to 1524 bytes of 
data plus an 8 byte Ethernet header. 

As each module 10 receives bytes (i.e. information 
packets, control characters, and tokens) from its predecessor 
in the ring, the module copies the bytes it receives internally 
and retransmits the bytes to a successive module 10 if 
appropriate. Bytes thereby flow at high speed around the 
ring architecture from one module 10 to another module 10. 
It is noted that tokens are only transmitted from one module 
to another module when a task initiated by the token has 
been completed. 

The ring architecture allows use of a variable number of 
modules 10. Four such modules 10, individually lOa-lOa", 
are shown in the particular embodiment described herein. It 45 
will be understood, however, that more modules 10 may be 
inserted into the ring architecture or that some of modules 
lOa-lOa* may be removed from the ring architecture. Thus, 
modules 10 "stack" to meet use requirements, e.g., system 8 
expands, to follow a growing user population or capacity 
requirement. 

Each module 10 communicates with two adjacent mod- 
ules through interconnecting communication links 12. Each 
of the links 12 is bi-directional For example link 12a goes 
from module 10a to 106 and link \2ab goes from module 
10b to module 10a. In normal operation the system uses 
links 12a, 126,12c, and 12 J. If one of these links is down 
(i.e. broken) the system automatically switches to links 12afc> 
to I2da. Such use of a set of backup links is conventional. 

Link 12d couples the output port 14a with the input port 
16b of module 106. Link 12b couples the output port 146 
with the input port 16c of module 10c. Similarly, link 12c 
couples the output port 14c with the input port 16d of 
module IGd. Finally, link I2d couples the output port 14d 
with the input port 16a of module 10a. Each communication 
link 12 is a high-speed communication path. The capacity 
for links 12 is established depending on the number of 
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modules 10 involved and the number of user devices 
attached to modules 10. In the specific embodiment shown 
herein communication links 12 operate at 300 Mbps. 

Modules 10 handle both synchronous data packets and 
asynchronous data packets. The synchronous packets 25 are 
fixed-length 16 byte packets 25. The asynchronous packets 
are variable length packets 35. Each module 10 has a 
number of asynchronous ports 30 (designated 30a to 30d) 
coupled to an asynchronous data buffer 32 and a number of 
synchronous ports 34 (designated 34a to 34a") coupled to a 
synchronous data buffer 36. Computers 18 are connected to 
asynchronous data ports 30 and telephones 20 and 22 are 
connected to synchronous data ports 34. 

Modules 10 are interconnected by links 12. Links 12 
establish a combined synchronous and asynchronous mes- 
sage transmission data network whereby computers 18 may 
share resources and data and whereby telephone conversa- 
tions may be conducted among the population of telephones 
20 and 22. 

Each module 10 has a timer 37 (individually identified as 
timers 37a to 37o) which controls the timing within the 
module. Each module also includes a conventional pro- 
grammed RISC processor and an associated memory which 
store and execute the programming operations described 
below. 

Each module also has a plurality of user devices con- 
nected thereto. As shown in FIG. 1, the user devices con- 
nected to the modules 10 include various computer work 
stations or terminals 18, analog telephones 20, and digital 
phones 22. 

Each analog telephone 20 is connected to a packet assem- 
bler and disassembler (PAD) 24. FIG. 2 illustrates in more 
detail a PAD 24. A PAD includes an analog-to-digital 
converter 24a, a sampling circuit 24b, a packet buffer 24c, 
and a packet assembler and disassembler block 24a*. Each 
PAD 24 produces a sequence of 16 byte packets 25 repre- 
senting voice sampled from a corresponding analog tele- 
phone 20. Such a sequence of packets carry "one side" of a 
telephone conversation. 

Each PAD 24 also receives a sequence of 1 6 byte packets 
25 for audible presentation of voice at the corresponding 
analog telephone 20. Block 24a* drives a digilal-to-analog 
converter 24e with incoming packets 25, i.e., the "other 
side" of a telephone conversation involving a user and 
telephone 20. Thus, block 24o* operates within a given 
module 10 providing and receiving packets 25 representing 
an analog telephone conversation and the associated normal 
in-band telephone signals. Packet 25 transport occurs at 64 
kbps in order to support the full duplex telephone traffic. 

Digital telephones 22 produce similar packets 25 repre- 
senting one side of a telephone conversation, i.e., voice 
sampled by digital telephone 22, and also receive a sequence 
of packets 25 for audible presentation at a digital telephone 
22. Digital telephones 22 exchange packets 25 with a 
module 10 at sufficient speed to support a full duplex 
telephone conversation, Le., 64 kbps. 

Telephone conversation data from telephones 20 and 22 is 
packetbed in the fixed-length 16 byte packets 25. The 
packets containing voice data must be delivered in a timely 
manner in order to maintain acceptable quality of voice 
communication. In order to accomplish timely delivery of 
data representing voice communications, all such data is 
handled by the present invention in a synchronous fashion. 
Since such data is handled in synchronous fashion conven- 
tional ''virtual circuits" can be established between user 
devices, e.g., between members of the population of tele- 
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phones 20 and 22. High quality telephone connections can 
therefore be achieved. There is no perceptible degradation in 
voice quality, because no more than about a four and a half 
millisecond delay exists in delivery of any given packet 25 
from end terminal to end terminal (i.e. from telephone to 
telephone). 

Computer work stations 18 produce data for delivery to 
other stations 18 and receive data from other stations 18. 
Because variation in delay and variation in packet size is 
generally acceptably in communications between stations 
18, such data is managed in an asynchronous fashion when 
transported via modules 10. Information exchanged among 
stations 18 is divided into "Ethernet" type packets, i.e., 
variable sized packets including addressing information 
according to an Ethernet type addressing schemes. 

The time frame for communication on links 12 is divided 
into a sequence of windows. FIG. 3 illustrates a sequence of 
windows 50, individually identified as windows 50a, SQb, 
etc. Each window 50 is two millisecond long 

Each window 50 begins with a "start window" control 
character or field 52 (which is one byte long). The start 
window control character indicates the onset of a window 
50. The remainder of each window 50 is dedicated first to all 
pending synchronous data transmissions and then to asyn- 
chronous data transmissions. More particularly, a first por- 
tion 54 of each window 50 is dedicated to exchange of all 
pending synchronous data packets 25. After all pending 
synchronous data packets 25 have been exchanged among 
modules 10, a second control character (not explicitly shown 
in FIG. 3) is transmitted around the ring to indicate the end 
of the synchronous transmissions. A second portion 56 of 
each window 50 is dedicated to exchange of asynchronous 
data packets 35. At the end of each window another control 
character (not explicitly shown in FIG. 3) is transmitted 
around the ring. As will be explained later, tokens and timing 
characters are also transmitted around the ring. 

The length of window 50 is two milliseconds long. The 
length of windows 50 is established by taking into account 
the bandwidth of the various communication paths. The 
length of window 50 is established so that all synchronous 
data can be delivered during each window and so that after 
the synchronous data is transmitted, sufficient reserve will 
remain in each window 50 to conduct exchange of asyn- 
chronous data. The actual allocation of a given window 50 
between synchronous and asynchronous data is dynamic. 
The allocation depends on the amount of pending synchro- 
nous data packets 25 which must be transmitted during the 
first portion of a given window 50. As the number of 
telephone conversations increases, the portion 54 of window 
50 used for such conversations increases. 

Thus the allocation between synchronous and asynchro- 
nous data in a given window 50 is not fixed but rather a 
function of the amount of synchronous data pending at the 
beginning of the window 50 with the remaining portion 56 
being used for asynchronous data. A control character 53 
which indicates that synchronous traffic is "all done" sepa- 
rates portions 54 and 56 of each window. This control 
character indicates the end of synchronous data transmission 
and the beginning of asynchronous data transmission within 
a given window 50. 

FIG. 4 illustrates, at a system level, data exchange during 
a given window 50. As shown in FIG. 4, processing loops at 
decision block 100 until start window control character 52 
appears on links 12. Processing then advances to block 102 
where modules 10 exchange all pending synchronous data 
packets, i.e., deliver all pending packets 25 in the synchro- 
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nous data buffers 36. Block 102 represents the overall 
exchange of all pending synchronous data packets 25 among 
modules lOa-lOd by ring message exchange. 

As indicated by decision block 106 a determination is 
made that all modules 10 have completed exchange of 
pending synchronous data, e.g., all modules 10 have deliv- 
ered all pending synchronous data packets 25. In other 
words, all synchronous data in buffers 36 at the onset of the 
current window 50 have been transmitted via links 12 to an 
appropriate module 10. At this point, communication among 
modules 10 switches from a synchronous mode of operation 
to an asynchronous mode of operation allowing variable 
length packets and an alternate addressing scheme. Asyn- 
chronous transmission is done using Ethernet packet rules 
and addressing codes to route the variable length packets to 
particular modules 10 and to corresponding user devices 
attached thereto. Ethernet packaging rules allow packets of 
varying length between 64 and 1524 bytes 

To prevent monopolization of window 50 by one module, 
each module 10 limits its use of portion 56 of each window 
so as to allow other modules 10 to deliver asynchronous 
data. Thus, block 108 represents delivery of a limited 
amount of asynchronous packets followed in decision block 
110 which tests use limitations. During block 108, a module 
10 sends a certain number of Ethernet packets to a succes- 
sive one of modules 10. Such module 10 limits its further use 
of the asynchronous portion 56 of a given window 50. That 
is, each module 10 is allowed a limited number of asyn- 
chronous data bytes per given window 50. In the embodi- 
ment described herein, each module 10 transmits a maxi- 
mum of 4000 bytes of asynchronous data in any given 
window 50. Thus, system level operation loops at blocks 108 
and 110 until all modules 10 have reached their use limit for 
the current window 50 or have delivered all pending asyn- 
chronous data. Processing eventually returns to block 100 
where system 8 waits for occurrence of the start window 
control character 52 and a next window 50. 

The modules 10 in general operate on a "peer" basis. 
However, one of modules 10 is given some degree of control 
over the process. In the embodiment shown, module 10a 
executes master control, that is, to some extent module 10a 
orchestrates the exchange of information on links 12 and it 
is in effect a timing master for the system. Modules 10 other 
than module 10a may be inserted and removed from the 
system as needed or desired without corrupting an overall 
control strategy, However, there must always be a control 
module 10a. 

Master control module 10a makes use of control charac- 
ters and tokens to orchestrate packetized information 
exchange within system 8. Modules detect the receipt of a 
token or control character by detecting one of the decodes of 
a byte other than the 256 data decodes. When a module 10 
receives a control character or a liming character, it imme- 
diately retransmits the control character or timing character 
to the next module 10. When a module 10 receives a token, 
the token is held until the module 10 is ready to retransmit 
the token, i.e. until the module is ready to relinquish its right 
to send packets of a particular type. 

FIG. 5 illustrates programming with respect to operation 
of module 10a. Module 10a is the master control module. As 
indicated by block 510, the process begins when module 10a 
transmits a "window start" control character 52 (see FIG. 3). 
Control character 52 is sent immediately around the ring 
architecture because when a module 10 receives a control 
character it immediately re-transmit (i.e. repeats) the control 
character. At this point, all modules 10 are prepared for the 
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onset of a window 50. As indicated by block 511 module 10a 
transmits all its pending synchronous packets 25, Le., mod- 
ule 10a empties synchronous data buffer 36a, into the ring 
on communication link 12a. Once module 10a has trans- 
mitted all of its synchronous data packets 25, as indicated by 
block 512 module 10a passes the "transmit synchronous 
packets" token to the next module, i.e., to module 106. 

After passing the transmit synchronous packets token to 
the next module on the ring, module 10a processes synchro- 
nous packet traffic from other modules until the transmit 
synchronous packets token is returned to module 10a. This 
is indicated by blocks 513 and 514. During this time, the 
remaining modules 10 will each in turn have opportunity to 
send all synchronous data packets 25 which were pending at 
the onset of the current window 50. That is, after module tOo 
receives the token, it transmits all its pending synchronous 
data packets 25, i.e., empties synchronous data buffer 36b, 
onto link 12ft. Module 10b then passes the token to module 
10c, giving module 10c an opportunity to send all its 
synchronous data packets 25 onto link 12c. The token is then 
passed to module lOo". When module lOd receives the token, 
it in turn submits all its synchronous data packets 25 which 
were pending at the onset of the current window 50 onto link 
12d As each module 10 submits its synchronous data 
packets 25 onto the ring architecture, each packet 25 reaches 
a target or module 10 which has attached thereto one of 
telephones 20 or 22 so as lo complete a virtual circuit. 

Eventually, all modules 10 will have had an opportunity 
to submit synchronous data packets 25 onto the ring and the 
"transmit synchronous packets" token will return to module 
10a. Processing then advances from decision block 513 to 
block 516 and module 10a sends the "all done synchronous 
data" control character 53 out on link 12a. Each of modules 
lQb-lOd thereby receive the "all done synchronous data" 
control character 53 indicating a transition from synchro- 
nous data exchange to asynchronous data exchange. As 
indicated by blocks 517 and 518, module 10a transmits 
asynchronous data packets 35 onto link 12a. As may be 
appreciated, each of the asynchronous data packets pass 
around the ring and eventually reach the intended 
destination, i.e., one of modules 10&-10d addressed as the 
destination address in the packet. 

Module 10a stops sending asynchronous data packets 
when one of two conditions is satisfied as indicated by 
decision block 517. Transmission of asynchronous packets 
by module 10a stops when module 10a determines that it has 
no more asynchronous data to transmit, (i.e., asynchronous 
data buffer 32 a is empty) or if module 10a has reached its 
use limit. In defined a module is limited to transmitting 4000 
bytes in one session. If module 10a has not reached its use 
limit and if there arc additional asynchronous data packets 
35 in buffer 32a, then processing returns to block 518 and 
module 10a continues to transmit asynchronous data packets 
Eventually, module 10a either reaches its use limit or 
exhausts pending asynchronous data in buffer 32a. Process- 
ing then advances to block 520 and module 10a passes the 
token to the next module, i.e., to module 10b. 

After passing the token to the next module, module 10a 
will process asynchronous data traffic as indicated by blocks 
521 and 522. Module 10a checks for return of the token as 
indicated by decision block 421. That is processing as 
indicated by blocks 521 and 522 continues until the token is 
returned to module 10a. When the token returns to module 
10a, ail modules 10 have had opportunity to send asynchro- 
nous data packets 35 onto the ring at least once up to their 
given limit, i.e., at least 4000 bytes. 

At this point, some portion of window 50 may remain. 
This is determined as indicated by block 523. If more time 
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remains module 10a can take advantage of this opportunity 
to send more asynchronous data packets 35. As indicated in 
FIG. 5 The "no" output from block 523 goes back to block 
517 and the process repeats. 

s Eventually, module 10a runs out of time in the current 
window 50 for transmission of additional packets 35. An end 
of ring control character is then transmitted around the ring. 
Processing then returns lo block 510 where module 10a 
again sends control character 52 and the process repeats. 

10 Module 10a also provides a timing reference for the 
system. Module 10a includes an interval timer 37a which 
produces interrupt signal every 15.625 microseconds. This 
liming reference signal is transmitted from module 10a to 
the other modules in the ring. When the timing interrupt 

15 occurs, module 10a transmits the special timing control 
character. The timing control character is inserted into any 
packet that is being transmitted at the lime the interrupt 
occurs. Thus, some synchronous packets 25 may be 17 bytes 
long after the clock control character is inserted therein. The 

20 additional delay introduced, i.e., a 16 byte synchronous 
packet 25 versus a 17 byte synchronous packet 25, does not 
introduce any noticeable delay to persons engaged in a 
conversation. The timing character is a 10 bit character 
which is not used for any other purpose and which each units 

25 recognizes as a timing character. When a unit on the ring 
(other than module 10a) delects this character, it repeats the 
character to the next unit on the ring and at the same time it 
re-synchronizes its internal clock 37. That is, the clock 37 in 
each unit is re -synchronized when the timing character is 

30 detected. In this way the clocks 37 in the various modules 
are kept in close synchronization. It is noted that as shown 
herein it is the control module 10a which introduces the 
timing character onto the ring, any one of the modules could 
perform this function. 

35 FIG. 6 illustrates the programming for modules 10 other 
than module 10a. (FIG. 5 shows the programming for 
module 10a). That is, FIG. 6 illustrates programming for 
modules lOft-lOdl The process starts as indicated by block 
610 when a module receives a "start window" control 

40 character 52. After receiving a "start window" control char- 
acter a module looks for a token with indicates that the 
module should start transmitting synchronous data. Process- 
ing continues iteration between blocks 611 and 612 until the 
module 10 receives the "start transmitting synchronous 

45 data" token. As indicated by block 611, when a module 
receives the "start transmitting synchronous data" token the 
module begins transmission of its synchronous data. Once a 
module 10 has transmitted all pending synchronous data 
packets 25 into the ring, it passes the "start transmitting 

50 synchronous data" token to the next module 10. This gives 
the next module 10 opportunity to submit its synchronous 
data packets 25. 

Once a module 10 has passed the "start transmitting 
synchronous data" the module processes synchronous 

55 packet traffic from other modules as indicated by block 615. 
Decision block 614 indicates that a module looks for "all 
synchronous traffic done" control character 53. Until the "all 
synchronous traffic done" control character 53 appears, 
processing iterates at blocks 614 and 615 and the module 

60 processes any synchronous data packets 25 appearing in the 
ring architecture from other modules. When the "all syn- 
chronous traffic done" control character 53 docs appear 
processing advances to block 616 and 617 where the module 
processes any asynchronous packet traffic appearing in the 

65 ring architecture from other modules. In decision block 616, 
module 10 looks for the "send asynchronous data" token. 
Processing iterates at blocks 616 and 617 until the module 
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receives the "send asynchronous data" token, "send asyn- 
chronous data" token is received asynchronous data packet 
35 are transmitted as indicted by block 620. After transmit- 
ting each asynchronous data packet 35, the module deter- 
mines as indicated by block 619 whether it has any addi- 
tional asynchronous data packets to transmit, or if the 
module has reached its use limit, e.g., has transmitted 4000 
bytes of asynchronous information in the current window 
50. Processing iterates at blocks 619 and 620 until no further 
asynchronous data packets remain or until the module 10 has 
reached its allotment or use limit allowed in the current 
window 50. Processing then advances to block 621 and the 
module passes the "send asynchronous data" token to the 
next module 10. 

After each of the modules 10 has had an opportunity to 
transmit both synchronous and asynchronous traffic during a 
particular window, there may still be time remaining in the 
window. When this condition occurs, the modules are given 
another chance to transmit additional asynchronous packets. 
This condition is illustrated in FIG. 6 by the path from block 
622 through block 625 to the entry of block 619. 

Following block 621, processing iterates between blocks 
622 and 623 where the module processes any further asyn- 
chronous packet traffic from other modules and looks for the 
occurrence of the end of window and the start new window 
control characters. If an end of window and start of new 
window control characters have not been received the pro- 
cessing goes from block 622 to 625 to 619. When and end 
of window and start of new window control characters arc 
received the processing returns to block 611. 

As an example of the capacity of the system, it is noted 
that a voice switching capacity on the order of 256 
simultaneous, full duplex calls may be implemented on a 
stack of eight modules 10. Each full duplex voice call 
consumes 64 kbps of data bandwidth. This translates into: 
(64,000 , 2)*256=32,768,000 or 32 Megabits (Mbps) of 
voice switching bandwidth on links 12 to support 256 
simultaneous full duplex voice calls. This is exclusive of 
framing overhead, which will be dependent on the hardware 
implementation. 

There are also other capacity considerations. Services 
which use 'redirection', e.g., voice compression, voice rec- 
ognition or fax services, all are very processor bandwidth 
intensive. This component creates an 'overhead factor* on 
the base voice switching bandwidth independent of framing 
overhead. Using an estimated overhead factor of thirty-three 
percent, (he voice switching bandwidth requirement 
increases from about 32 Mbps to about 44 Mbps. 

For asynchronous data capacity, a minimum carrying 
requirement of a single 100 Mbps Ethernet will meet a given 
level of computer network use expectations. This brings the 
aggregate carrying capacity for system 8 to about 144 Mbps. 

For control and management an overhead of not more 
than about 4 Mbps per module 10 is expected. Of this, 1 
Mbps is reserved for true intcr-module 10 communication, 
0.5 Mbps is reserved for network management, 1.5 Mbps is 
reserved for call accounting and 1 Mbps is reserved for event 
logging and tracing. For eight modules 10 in a system 8, a 
subtotal of (8*4,000,000) or about 32 Mbps. This brings the 
entire switching capacity requirement for system 8 to about 
176 Mbps. Establishing a 300 Mbps capacity for links 12 
supports this expected switching capacity. 

Naturally, other systems could be implemented using the 
present invention with different requirements and capacity 
considerations. For example various numbers of modules 10 
could be connected in a ring utilizing the present invention, 
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While the invention is described as applied to a LAN 
environment, it is noted that the invention could also be 
applied in a WAN environment It is also noted that the 
communication path between modules 10 could be either 
electrical or optical without departing from the present 
invention. 

It should be appreciated that the computer data commu- 
nicated between modules 10, could include all the various 
kinks of data normally transmitted between computer. For 
example such data could include video data and 
PC-telephone data. 

A combined synchronous and asynchronous message 
transmission method and apparatus has been shown and 
described. The integration of synchronous and asynchronous 
message transmission into a single communication system 
provides opportunity for integrated communication services 
incorporating both computer data and voice data. Despite 
integration of synchronous and asynchronous data, synchro- 
nous data arrives in timely fashion without degradation in 
voice quality. . 

As described herein data from computer 18 is treated as 
asynchronous data. However, under certain conditions, com- 
puters 18 may be required to deliver time -sensitive infor- 
mation in a synchronous manner, and could be treated as 
such by a corresponding module 10. 

System 8 also includes an external connection 38, e.g., a 
high speed telephone or network connection, whereby other 
systems may introduce information into system 8 or take 
information from system 8. Such links are handled similar to 
links directly connected to modules 10. 

It is noted herein that a single asynchronous protocol is 
used. However, additional control schemes may also be 
employed. For example such protocols could be used to 
allow further exchange of asynchronous data when modules 
10 have reached their use limit for asynchronous data, but 
window 50 has not yet expired. 

It will be appreciated that the present invention is not 
restricted to the particular embodiment that has been 
described and illustrated, and that variations may be made 
therein without departing from the scope of the invention as 
found in the appended claims and equivalents thereof. 

What is claimed is: 

1. A communication system comprising: 

a plurality of communication modules interconnected to 
exchange information there between, 

said information including a variable amount of synchro- 
nous information and a variable amount of asynchro- 
nous information, 

said modules providing and receiving said synchronous 
information in substantially fixed length packets, said 
modules providing and receiving said asynchronous 
information in variable length packets, each of said 
packets comprising a plurality of multi bit bytes; 

a communication channel coupling said modules together, 
operation of said communication channel being divided 
into repealed fixed length windows, said channel oper- 
ating in accordance with a token passing protocol, 

each module transmitting a variable number of fixed 
length packets between said modules during a first 
portion of each of said fixed length windows and each 
module transmitting a variable number of variable 
length packets between said modules during a second 
portion of each of said repeated fixed length windows, 

each module transmitting a token to the next module 
when that module has completed the transmission of all 
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synchronous packets from thai module to other 
modules, in order to indicate that the next module can 
begin transmission of synchronous packets, and 

a timer which periodically inserts a timing byte into the 
string of bytes being transmitted between said modules s 
on said communication channel, each of said modules 
including a timer which is rcsynchronized in accor- 
dance with said timing byte. 

2. A system for transmitting synchronous and asynchro- 
nous information, said system comprising: 10 

a plurality of modules coupled in a ring, each module 
receiving input from a predecessor module and provid- 
ing output to a successor module, each of said modules 
referencing a repeating sequence of time windows, 
each time window including a first portion dedicated to 15 
exchange of synchronous data and a second portion 
dedicated to exchange of asynchronous data, said ring 
operating in accordance with to a token passing 
protocol, said synchronous data being in the form of 


12 

fixed length packets and said asynchronous data being 
in the form of variable length packets, 

said first portion of said time window terminating when 
all pending synchronous data has been exchanged 
among said plurality of modules, each module trans- 
mitting a token to the next ring when it is completed 
transmitting all synchronous data pending in that 
module, use of said second portion of said time window 
by each module being limited whereby each of said 
modules has opportunity during said second portion of 
said window to transmit some asynchronous data, 

each module including a clock, synchronization between 
said modules being maintained by periodically trans- 
mitting a character on said ring which is recognized by 
each module as a timing character and wherein each 
modules resynchronizes its clock when said character is 
received. 

♦ * * * ♦ 
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[57] ABSTRACT 

An asynchronous, fiber optic, ring network includes a 
number of nodes where data enters and exits the net- 
work. Each of these nodes has its own clock to provide 
timing needed by equipment at that node. So that syn- 
chronous data can be transmitted between predeter- 
mined nodes, a master node provides timing informa- 
tion which may be used at any node to synchronize its 
clock. Timing information comprises the propagation 
delay around the ring (ring latency) as measured by the 
master node. This measure of propagation delay is 
transmitted by the master node as an information packet 
available at all nodes. Synchronization of any other 
node with the master requires that the other node per- 
form its own measurement of propagation delay, com- 
pare its measurement with that of the master node, and 
make adjustments to its own clock that tend to decrease 
the difference between these measurements. 

13 Claims, 4 Drawing Sheets 


r 600 

CLOCK RECOVERY 



COUNTER 

START 


STOP 

CLOCK 

RESET 



OUTPUT 



jrMO 


r630 


LOW-PASS 
FILTER 


r650 

PHASE 8 
COMPARATOR i 


VOLTAGE 
CONTROLLED 
OSCILLATOR 
(N X 6 khz) 


^90 


j-50 


04/01/2004, EAST Version: 1.4.1 


U.S. Patent Apr. 27, 1993 sheet 1 of 4 5,206,857 



04/01/2004, EAST Version: 1.4.1 


U.S. Patent 


Apr. 27, 1993 


Sheet 2 of 4 


5,206,857 



04/01/2004, 


EAST 


version: 


1.4.1 


U.S. Patent Apr. 27, 1993 sheet 3 of 4 5,206,857 


FIG. 3 



h 9000 SYMBOLS MAXIMUM — H 

/310 

PREAMBLE 

SO 

FC 

OA 

SA 

INFO 

FCS 

EO 

FS 


-240 


FIG. 4 


P/0 MEDIA ACCESS CONTROLLER 
| REPEAT 


IDLE 
SOURCE 


PACKET 
SOURCE 


TOKEN 
SOURCE 


FROM NETWORK 


/ INFO / FCS / 
X 310 


ft / FC / SO 



TO NETWORK 


-240 


FIG. 5 


P/0 MEDIA ACCESS CONTROLLER 



FROM NETWORK 


/ IDLE /SA/DA/FC/ J > 


TO NETWORK 


04/01/2004, EAST Version: 1.4.1 


U.S. Patent Apr, 27, 1993 sheet 4 of 4 5,206,857 



04/01/2004, EAST Version: 1.4.1 


5,206,857 

1 2 

synchronous information over an asynchronous facility 

APPARATUS AND METHOD FOR TIMING is the use of elastic storage registers to buffer the differ- 

DISTRIBUTION OVER AN ASYNCHRONOUS cnces in the bit rate. That is, data are written into a shift . 

WNG register at a first bit rate and read out of the shift register 

_-„ uiaT « AT cipin 5 8t 8 ^ ond ttt rate. When racket information is being 

ti^imuAL rifiLU transmitted, it makes little difference whether the read 

This invention relates to a communication network and write rates are slightly different because packets are 

having a plurality of nodes, and more particularly to a generally limited in size and the elastic storage registers 

technique for distributing timing synchronization can be made as large as desired. However, when trans- 

among selected nodes. mitting continuous synchronous data, the elastic storage 

BACKGROUND OF THE INVENTION registers will overflow or underflow with the undesir- 

. . , . . . . able result that transmitted information will either be 

Surprisingly, as computers grow in both power and Iost (overflow eo^on), or that incorrect information 
proliferation, so too > does their need to borrow and wui ^ created (underflow co^^). 
share more data with other computers. This need to 15 TT c n v. \ Q ,, . ' e t . ,_ 0ft 
exchange greater amounts of information can no longer "f. Pal No ' ™ 00 Sep. 12, 1989 
be fully 'satisfied by the periodic data transfer between ™ 6 chsc5os f a flbcr -°P tlc voice/data network. This 
two computers but, rather, requires the simultaneous Pf lent tcachcs a tcchmc l uc for synchronizing a local 
interconnection among a number of them, each having c,ock bv monitoring thc average fill of an elastic storage 
a particular specialization yet drawing from the special- » register (receiving buffer), speeding up the local clock 
ization of the others. These interconnections are known whcn the average fill is increasing, and slowing the 
as networks, and while they are limited in size and clock when the average fill is decreasing so that over- 
found in only corporate environments today, vast flow and underflow are prevented. While this technique 
global geodesic networks will connect millions of is- is useful, it requires that synchronous data be continu- 
lands of information tomorrow. 25 0 usly present to maintain synchronization. 

Communication systems which allow data transfer It is also known to distribute a reference timing signal 

over telephone lines at a few hundred bits per second 0VC r a separate link to each node in a network, includ- 

have been an integral part of computer systems for the j ng a ring network. However, such a technique requires 

past few decades. Only recently have networks capable thc installation of a separate network just for riming— 

of handling several million bits per second been widely 30 ^ dcfeati the strucluraI simpIicity of ^ ^ md 

available Local ^networks LAN) have typically flddi |0 fts C0SI , fc ^ ^ £ 

offered between 100 Kb/s and 10 Mb/s among a few nMn 7. .„ _ f ,. ., . . . K e 

hundred stations, and have been limited to a local area Upon pn0r art ****** for Routing timing infor- 

(a kilometer or so). One such network, Ethernet, is 1,1311011 over synchronous nng. 

synchronous and operates at 10 Mb/s. Because rapid 35 SUMMARY OF THE INVENTION 

information transfer is indispensable in our highly com- ... 

petitive society, Ethernet is being superseded by a In accordance with the invention, a ring network 
higher capacity network knwon as the Fiber Distrib- includes a master node and one or more slave nodes 
uted Data Interface (FDDI) which transmits 100 Mb/s where data enter and exit the network. Each of these 
of data over each of two counter-rotating rings. FDDI 40 nodes includes a clock for supplying timing information 
can tolerate a separation of up to 2 kilometers between to equipment at that node, The master node measures 
stations, and support and a total cable distance of 100 the time delay encountered by data traversing the net- 
kilometers around a ring with 500 station attachments. work, and transmits this measurement to the slave node. 
FDDI possesses enough bandwidth to support up to 800 At the slave node, a similar measurement of time delay 
voice channels or perhaps 1-2 digitized video channels. 45 is made and compared with the measurement transmit- 
One problem with voice or video traffic over FDDI, ted by the master node. Differences between these two 
however, is that the network and interface are asyn- measurements are used to adjust the frequency of the 
chronous, thereby preventing timing information from clock at the slave node 

passing across the network boundaries. Although In the Ulustrative embodiment of the invention, clock 

FDD! uses a Tuned Token Protocol to provide both 50 syiJchronization j, optionalJv m6 bdependently avail- 

synchronous and asynchronous service no technique able tQ ^ m ^ ^ nctWQrL ^ ^ 

has emerged as being clearly superior for clock syn- ^ simultaneously supports both synchro- 

chromzation at various stations around the nng. * . . 7 " V , 

Whereas packet switching is possible over an asynchrcl n0US Smd »y»chionous data traffic. Accordingly the 

nous network, circuit switching requires a synchronous 55 P™*** SS/™ ^toP"* a synchronous FDDI 

network and for that reason a synchronous FDDI net- BCtwork 0™MD no longer obtains, 

work (FDDI -II) has been proposed. Unfortunately, BRIEF DESCRIPTION OF THE DRAWING 

FDDI and FDDI-II are incompatible, which is to say _ _ 

that a node adapted for FDDI-II operation cannot be ™ e features ^ capabilities of the present invention 
pan of an FDDI ring. a will be more fully understood when reference is made 
In an asynchronous system, each link requires its own t0 thc detailed description and the drawing, of which 
clock. This means that each link is frequency and FIp- 1 illustrates a ring network haying counter- 
phase-asynchronous vis-a-vis all other links in the ring, rotating rings and a plurality of nodes, each serving 
and that timing information cannot be recovered from different host system equipment; 
the bit stream. Accordingly, the distribution of synchro- 65 FIG. 2 discloses equipment used at a particular node 
nous information, such as conventional telephone con- in accordance with the invention; 
versations, on the FDDI system, has certain inherent FIG; 3 shows a data packet such as used in the Fiber 
problems. One well-known solution for transmitting Distributed Data Interface (FDDI); 
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FIG. 4 illustrates the operation of a Media Access 
Com roller at a node when repeating a data packet to the 
network; 

FIG. 5 illustrates the operation of the Media Access 
Controller at a node when supplying an idle data packet S 
to the network; and 

FIG. 6 discloses, in block form, the details of clock 
recovery in accordance with the invention. 


DETAILED DESCRIPTION 
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An emerging standard (ANSI X3T9.5)— better 
known as the Fiber Distributed Data Interface (FDDI- 
) — defines a 100 Mb/s, time-token protocol. This proto- 
col is to be implemented on an asynchronous, fiber- 
optic network optimized for 1300 nanometer technol- 15 
ogy. Fiber-optic communication is best suited for point- 
to-point transmission end is present in two local area 
network (LAN) topologies, the active hub star, and the 
ring. Active stars require extensive connections to 
wherever the central hub is located and introduce a 20 
single failure point that can disable the entire LAN. 
Single-ring networks are like chains which are vulnera- 
ble to failures at any station. FDDI minimizes this by 
using dual, counter-rotating rings. This provides an 
alternate path if a station or a link fails, allowing data to 25 
be wrapped back onto the secondary ring in the event of 
failure. The secondary ring may be only a standby ring, 
or it may be used for concurrent transmission thereby 
providing a 200 Mb/s network. Additional background 
information regarding FDDI is contained in an article 30 
by Floyd E. Ross entitled: FDDI— a Tutorial, published 
in the IEEE Communications Magazine, May 
1986- Vol. No. 24, No. 5 at pp. 10-17. 

Referring now to FIG. 1, there is disclosed an exam- 
ple of an FDDI network comprising primary ring 10 35 
and counter-rotating secondary ring 20. Nodes are illus- 
tratively positioned at locations around the network 
that may be separated by large distances. Associated 
with each node is a host system that is available to users 
at all other nodes via the fiber-optic ring. Together they 40 
form a network whose combined resources are formida- 
ble and diverse. A generalized host system 100 is shown 
connecting stations 101, 102 to the FDDI network via 
network interface (N/1) 200. In the discussion that fol- 
lows, network interface 200 and host system 100 are 45 
designated as the "master node". In the present inven- 
tion, the master node distributes timing information to 
all of the other nodes. Accordingly, network interface 
200 is similar to the other network interfaces 300 in all 
respects except that it supplies timing information to SO 
them. Network interface 200 receives a timing signal 
over line 50 via host system 100. Any pair of nodes 
desiring to exchange synchronous data with each other 
need to synchronize their clocks. This can be accom- 
plished by adjusting the clock frequency at one of the 55 
nodes to agree with the other, or by adjusting the clock 
frequency of both of the nodes to agree with a reference 
clock. Once synchronization is achieved, information 
such as digitized voice can be transmitted and received 
by appropriate equipment at these nodes. 60 

Host system 100 can be any mainframe, workstation, 
minicomputer or peripheral to which a network inter- 
face is attached. Its role is to provide and receive net- 
work data and perform some higher-level protocol 
functions which are not handled by the associated sta- 65 
tions. Examples of other host systems include, but are 
not limited to, mainframe 110, PBX 120, and gateways, 
130, 140. 


Stations 131, 132 are shown connected to gateway 
130 by, for example, a network 30 (e.g., IEEE-802.5 
token ring). Stations 141, 142 are shown connected to 
gateway 140 by, for example, a bus 40 (e.g., IEEE-802.3 
Ethernet or IEEE-802.4 token bus). Stations 121, 122 
are shown connected to PBX 120. Each of these nodes 
connects to the fiber-optic ring by a network interface 
(N/I) 300 whose operation is defined in detail by FDDI 
standards, but will be briefly discussed hereinafter. 

FIG. 2 discloses the equipment needed at each node 
to support interconnection with the FDDI network. 
Optical fibers 10, 20 are made from glass-clad silica and 
consist of an inner core surrounded by a glass cladding 
(having a different refractive index than the silica) and 
a protective sleeve. Plastic cladding is not used in 
FDDI applications because of its greater attenuation. 
Optical receivers 261, 281 receive modulated light- 
wave signals, whose center wavelength lies between 
1270 and 1380 nanometers, from optical fibers 20, 10 
respectively, and consist of a photodetector, an ampli- 
fier, a shaping filter, a comparator, a buffer to provide 
suitable electrical levels, and a signal detect circuit. The 
photodetector is constructed from an InGaAs/lnP PIN 
photodiode that senses light intensity and converts it 
into current pulses. Optical transmitters 251, 271 trans- 
mit modulated light-wave signals, whose wavelengths 
are also in the 1300 nanometer range, to optical fibers 
10, 20 respectively. Each optical transmitter consists of 
a driver and a light emitting diode (LED). The LED is 
typically made of InGaAsP, and generates light whose 
intensity is a function of the input data to be transmitted. 

Transmitter/receiver pair 250, 260 controls the en- 
coding and decoding of data and control symbols, seri- 
alizing the data clock recovery, line-state detection and 
reporting. In operation, transmitter 250 serializes eight- 
bit-wide parallel data from Media Access Controller 
(MAC) 240 to produce non-return-to*zero (NRZ) code. 
4B/5B coding is used which comprises translation of 
4-bit groups of data into a 5-bit value, and conversion 
into the NRZ format for transmission on the fiber-optic 
ring. 4B/5B encoding is said to be 80% efficient since a 
100 Mb/s data rate translates into a 125 megabaud rate 
which is present on the fiber-optic ring. Receiver 260 
decodes the data received from the fiber-optic ring and 
converts it into symbols that can be recognized by the 
MAC Tasks of receiver 260 include retiming the data 
to an internal clock via a phase-locked loop and an 
elastic buffer, and converting 5-bit code back into 4-bit 
code. Differential Manchester coding is frequency com- 
pared with 4B/5B coding and has the advantage of 
being rich in clock information with a transition at 
every bit, but the disadvantage of being only 50% effi- 
cient. Suitable devices for the transmitter and receiver 
are the AM7985 and AM7984, respectively, which are 
commercially available from Advanced Micro Devices, 
Inc. Transmitter 270 and receiver 280 are identical to 
transmitter 250 and receiver 260, but are dedicated to 
different optical fibers. 

Bus 60 comprises several buses. It includes, for exam- 
ple, a 32-bit bus used to interconnect buffer memory 230 
with other devices, while several 16-bit buses are used 
between controllers 210, 220 and 240 and node proces- 
sor 103. 

Data Path Controller (DPQ 210 converts data in 
received packets from byte-wide to 32-bit parallel word 
formats, performs parity checks and generates packet 
and node status. A suitable device for the DPC is the 
AM79C82. 
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RAM Buffer Controller (RBC) 220 generates ad- mum allotment of synchronous data, the negotiated 
dresses to buffer memory 230 for received and transmit- target token-rotation time (TTRT) will not be ex- 
ted packets. The RBC handles buffer management and ceeded. The transmission of asynchronous data is 
arbitrates direct memory access coming from DPC 210, slightly more complicated, since the maximum time 
node processor 103 and other host system apparatus. A 5 allotted for transmission is not constant. Instead, a sta- 
suitable device for the RBC is the AM79C81. tion can transmit asynchronous data until the unused 

Buffer memory 230 is a conventional 256-kbyte static bandwidth on the ring is exhausted, 

random access memory which is readily available from MAC 240 controls the flow of data transmission on 

a number of commercial sources. The buffer memory is the ring, and indicates when a data packet is transmitted 

a storage area accessible by the controllers 210, 220, 10 as well as when it has returned from a trip around the 

240, the node processor 103 and other host system network. This feature is highly important in the practice 

equipment. of the invention as will be pointed out later. FIG. 3 . 

Media Access Controller (MAC) 240 controls the discloses the format of an FDDI packet 310. Packets are 
right to transmit data to the network based on capturing preceded by a PREAMBLE having a minimum of 16 
a token according to a set of network rules, recognizes IS "idle" control symbols. Packets begin with a Start De- 
addresses, controls ring recovery, and handles network limiter (SD) composed from the J and K control sym- 
and frame status. A suitable device for the MAC is the bob of the 4B/5B code. This is followed by a 2-data- 
AM79C83. The above devices designated 210, 220, 240, symbol Frame Control (FC) field that identifies the type 
250, 260 are all commercially available from Advanced of packet. The Destination Address (DA) identifies the 
Micro Devices, Inc. in what is known as the SUPER- 20 intended recipient of the packet. Likewise, the Source 
NETTM chip set which has been designed to conform Address (SA) identifies the packet's sender. Addresses 
to the FDDI standard. can be either 26 or 48 bits in length. The DA field can 

Node processor 103 is a separate microprogrammed point to a single station, a group of stations, or all sta- 

or conventional microprocessor-based system used for lions on the ring. Following SA comes the variable 

offloading the host .and overseeing the operation of the 25 length information field (INFO), A frame Check Se- 

network interface 200. The node processor communi- quence (FCS) field contains 4 bytes of data that are the 

cates with the network interface over bus 60. A suitable result of a 32-bit Autodin II cyclic redundancy check 

device is the AM29000 RISC Processor, also commer- polynomial. The FCS ensures the data integrity of the 

cially available from Advanced Micro Devices, Inc., FC, DA, SA, INFO, and FCS fields. Following the 

which includes a number of embedded counters. One of 30 FCS field, an End Delimiter (ED) formed with the "T" 

these counters is used as counter 660 in FIG. 6, and is symbol is transmitted. The Frame status (FS) field is 

shown functionally separated from the node processor used for symbols that determine whether the packet 

for greater clarity in explaining the invention. was received with error, if the address was recognized, 

Block 104 (Synchronous and Asynchronous Inter- or whether the packet was copied. The maximum 

face) contains the circuitry needed to buffer information 35 length of the packet (here 9000 symbols) is limited by 

between memory 230 and a synchronous Time Division the length of the elastic buffer and the worst-case fre- 

Multiplexed (TDM) bus. Here, terminal 101 and tele- quency difference between the clocks at two nodes 

phone 102 represent the kinds of devices that might be Referring now to FIG. 4, it will be seen that MAC 

connected to such a bus. Block 104 uses Direct Memory 240 acts like a switch. Normally the switch selects a 

Accessing to move data between memory 230 and an- 40 source of IDLE control symbols for transmission on the 

other buffer memory within block 104. In addition, ring. When the Start Delimiter arrives, MAC 240 

block 104 contains the circuitry needed to move data to switches to a repeat path. The packet is monitored, 

and from bus 60 and the TDM bus. A source of syn- copied if required, and simultaneously repeated. MAC 

chronous timing is provided to block 104 over line 90. 240 can also switch to source its own packets or issue a 

Since the present invention is concerned with establish- 45 token. Packets are removed from the ring by the origi- 

ing timing, such as present on line 90, rather than its use, nating station. This process, called "stripping", is illus- 

further details regarding block 104 arc omitted for clar- trated in FIG. 5. The MAC repeats the packet until the 

ity. SA field is received. Upon recognition of the Source 

FDDI Protocol Address, the switch moves to the IDLE position. The 

50 resulting packet fragment on the ring is ignored and 

Information is transmitted on the FDDI ring as pack- eventually removed when it reaches a station holding 

els of data, each having a maximum size of 9000 symbols the token for transmission. A station wishing to transmit 

(4 bits/symbol) and only one node may transmit a data must first "capture" a token. The token is a 6-symbol 

packet at a time. However, in order to avoid anarchy, a packet that is uniquely recognized. A station captures 

single token is passed from node to node giving the 55 the token by performing the stripping action. Only the 

token holder an exclusive right to transmit data. FDDI token SD field is repeated on the ring. Once the token 

protocol promotes fair an deterministic access to net- is captured, the station can begin transmitting packets, 

work resources for all stations. This is done using a When the last packet is sent, the station immediately 

timer that measures the time between token arrivals, a follows by issuing a new token, 

timer that controls how long a token can be held for 60 It is noted that FDDI is asynchronous, which means 

transmission, and a counter that indicates the number of that the clock at each of the various nodes is not syn- 

times the token arrives later than expected. The rules chronized with a common oscillator. Such synchroniza- 

for data transmission on FDDI depend upon the type of tion frequently comes at a very high price. It is impor- 

data to be sent. For transmission of synchronous data, tant, however, that each of the nodes be equipped to 

the rules are straightforward. The amount of synchro- 65 receive and transmit data at an agreed-upon rate, and 

nous data that can be transmitted at a given token op- that data formats be established for efficient communi- 

portunity is limited by the bandwidth-allocation process cation. FDDI employs an interesting solution to the 

which assures that if every station transmits its maxi- ring clocking problem. The total ring, including all of 
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its stations and all of its links, must continue to have the provides an output state that neither increases nor de- 
same apparent bit length during data transmission. Oth- creases the frequency of the VCO. When the "A" and 
erwise, some bits would be lost or gained as a frame was "B" inputs to the phase comparator are identical, the 
repeated around the ring. In the face of jitter, voltage, VCO is at the desired frequency, although the output 
temperature, and aging effects, such stability can only S signal alternates between the two logic levels. Low- 
be realized through special provisions. At each node, an Pass filter 630 is used to smooth such time-varying sig- 
elastic buffer is inserted between the receiver, which nals at the input to VCO 640 so that an averaged volt- 
employs a variable frequency clock to track the clock of age is presented to the VCO. 

the previous transmitting station, and a transmitter, Synchronizing VCO 640 to timing information sup- 

which runs on a fued -frequency clock. The elastic 10 plied by a remote source of tuning is accomplished 

buffer at each station, is re-initialized during a preamble when switch 625 is in position 3 where operation as a 

which precedes each frame or token. This has the effect . slave node is desired. The fundamental principle 

of increasing, or decreasing, the length of the preamble adapted for use in the present invention is simply stated: 

which is initially transmitted as 16 or more symbols. independent observes measuring the identical phenome- 

The transmitter clocks typically have 0,005% stability 15 non should reach an identical result If not, then the 

and the elastic buffer stores 10 bits of data. Accord- measuring apparatus is modified until the results are 

ingly, while significant data can be transmitted without identical. In the present invention, the phenomenon to 

exceeding the limits of the elastic buffer, practical trade- be measured is the time delay that a data signal encoun- 

offs are made between the accuracy of the clock and the ters during a single trip around the ring network. Since 

length of the buffer. 20 the ring network is identical for all observers, then they 

FIG. 6 discloses detail regarding the construction of should all measure identical time delays— but they do 
clock recovery circuit 600 which is used at both master not. And the reason that they do not is because the 
and slave nodes to provide a source of timing on line 90 clocks that are used to make the measurements are dif- 
to all equipment at the node which it serves. Switch 625 ferent. The present invention designates one observer 
is used for selecting between operation as a master tim- 25 (master node) to make the "official" measurement of 
ing source (position 1), a free-running timing source time delay around the ring (ring latency), and to com- 
(position 2), or a slave timing source (position 3). Be- municate this measurement to all nodes. Any node seek- 
cause digital voice communication equipment at tele- ing to become synchronous with the master node makes 
phone offices typically use 8 kHz as the sampling fre- its own measurement of ring latency, compares it with 
quency when carrying out analog-to-digital conversion, 30 the "official" measurement made by the master node, 
8 kHz is the information rate for synchronous communi- and adjusts its clock so that its measurement of ring 
cations. Accordingly, voltage-controlled crystal oscilla- latency is identical with that of the master, 
tor (VCO) 640 operates at a multiple (N) of the 8 kHz Returning now to FIG. 6, adjustments to VCO 640 
base frequency so that it can be easily synchronized are made based on differences in ring latency measure- 
therewith. In an illustrative embodiment, VCO 640 35 ments. At this time, switch 625 is in position 3 (Slave 
operates at 8.192 MHz, and N= 1024. Synchronization Mode). AND gate 611 responds to three input signals 
is not required at any node which operates asynchro- from line 70 to provide a start pulse for counter 620. 
nously, although synchronization requires very little in When a synchronous data packet leaves Media Access 
the way of additional components. For example, in the Controller 240 (see FIG. 2), all three inputs to AND 
preferred embodiment of the invention counter 620 is 40 gate 611 are high causing its output to also become high, 
actually part of node processor 103 (see FIG. 2), and Using the illustrative AM79C83 device, these leads are 
gate array 610 only adds a few gates to a larger gate designated RDYTBYT, XFRBYTE and XMEDAVS. 
array (not shown) used for performing a number of It is noted that this synchronous packet is generated by 
other tasks within the host system. In the Free-Run the node, and not a repeated synchronous packet! 
Mode, (position 2 of switch 625) VCO 640 provides an 45 Another signal lines FSVLD*, also from MAC 240, 

8.192 MHz crystal-controlled timing signal to all using signals when the node has received its own packet 

equipment. VCOs are well-known devices that are which is to say that the transmitted packet has traversed 

available from a number of vendors including CTS the entire ring and needs to be stripped from the ring. 

Corporation and SaRonix. This signal (FSVLD*) is used to stop counter 620 so 

Synchronizing VCO 640 to an 8 kHz signal, such as 50 that the number which is present at the output of the 
provided by a telephone office, is accomplished when counter is the number of cycles of the 8.192 MHz clock 
switch 625 is in position 1 where operation as the master that have occurred between the start and stop pulses 
source of timing for the ring network is desired. Divide- and is a measure of ring latency delivered to node pro 
by-N counter 660 functions to divide the 8.192 MHz cessor 103 (see FIG. 2) which appends this number to 
signal by 1024 to provide an 8 kHz square wave whose 55 the end of the next synchronous packet that is to be sent 
phase is compared with the phase of a reference 8 kHz around the ring. This is done regardless of whether the 
signal on input line 50. Phase comparator 650 is a con- node is the master or a slave. If the node is the master, 
ventional phase-locked -loop, such as the 74HC4046M, the process is ended. If the node is a slave, the node 
which is a commercially available device from a num- processor compares the value that is read from counter 
ber of vendors including Signetics and National Semi- 60 620 to the value that the master node transmitted on its 
conductor Corporation. The output of the phase com- last synchronous packet. These values are subtracted 
parator is a measure of the error in the frequency of the from each other in the node processor and the results of 
VCO. This error signal comprises a binary signal whose the subtraction are delivered to latch 615. In the pre- 
logic levels are used to increase/decrease the frequency ferred embodiment of the invention, latch 615 provides 
of VCO 640. Alternatively, a phase comparator having 65 three output states comprising +5 volts, ground, or an 
output states other binary voltages may be advanta- open circuit. These states are used to either increase, 
geously used in the present invention. Indeed, the fol- decrease, or not affect the frequency of the voltage-con- 
lowing paragraph discloses a Tri-State Latch 615 that trolled crystal oscillator 640. 
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Although a particular embodiment has been shown 
and described, it is understood that various modifica- 
tions are possible within the spirit and scope of the 
invention. These modifications are possible within the 
spirit and scope of the invention. These modifications 5 
include, but are not limited to, use of the invention on 
non fiber-optic networks, ring latency measurements in 
which the data packet makes more than a single excur- 
sion around the ring, use of the invention in a network 
where packets, per-se, are not used, and use of the in* 10 
vention in configurations where node connections are 
parallel rather than serial. Furthermore, it is understood 
that in a network of counter-rotating rings, either ring 
may be used for ring latency measurements and either 
ring may be used to transmit the results of such mea- is 
surements. While it is preferred that the master and the 
slave nodes each use the same ring to make latency 
measurements, it is not required. 

I claim: 

1. A communication network interconnecting a mas- 20 
ter node and at least one slave node in a ring topology, 
each node including a clock for providing timing signals 

to equipment at thai node, each node further including 
means for transmitting and receiving data to and from 
the ring network, 25 

characterized in that: 

the master node includes: 

means responsive to the clock at the master node for 
measuring a first time delay encountered by data 
traversing the ring network; 30 

means for transmitting the measure of the first time 
delay to the slave node, 

the slave node includes: 

means responsive to the clock at the slave node for 
measuring a second time delay encountered by data 35 
traversing the ring network; 

means for receiving the measure of the first time 
delay from the master node; and 

means responsive to the difference between the mea- 
sured first and second time delays for adjusting the 40 
frequency of the clock at the slave node to bring its 
frequency into a predetermined relationship wiih 
the frequency of the clock at the master node. 

2. The invention of claim 1 wherein the measure of 
the first time delay is transmitted to the slave node over 45 
the ring network. 

3. The invention of claim 1 wherein time is measured 
by counting the number of clock cycles that occur be- 
tween the time when a data packet enters the ring net- 
work and the time when that packet exits the network, 50 
said data packet being transmitted and received by node 
performing the measurement. 

4. The invention of claim 1 wherein the predeter- 
mined relationship between the frequency of the clock 

at the slave node and the frequency of the clock at the 55 
master node is equality. 

5. The invention of claim 1 wherein the communica- 
tion network comprises fiber-optic links between the 
nodes. 

6. The invention of claim 5 wherein the communi- 60 
cation network conforms to the Fiber Distributed Data 
Interface protocol. 

7. A local area network, configured as a token -pass- 
ing ring, for communicating packets of information 
between nodes attached to the ring, at least one of the 65 
nodes being designated slave for the purpose of receiv- 
ing information regarding the frequency of the clock at 
the master node, the slave node 


10 


characterized by: 
means for transmitting an identifiable packet onto the 
ring; 

means for receiving said identifiable packet after it 
has traversed the ring; 

means responsive to a clock at the slave node for 
measuring the time interval between the event that 
the identifiable packet is transmitted and the event 
that it is received in order to measure ring latency; 

means for receiving a measurement of ring latency 
made at the master node; 

means jointly responsive to the ring latency measure- 
ment made at the master node and to the ring la- 
tency measurement made at the slave node for 
modifying the frequency of the clock at the slave 
node in a manner that decreases the difference 
between the latency measurements. 

8. A method for timing synchronization in an asyn- 
chronous ring network that interconnects first and sec- 
ond stations; each station having a timing signal genera- 
tor, a transmitter for entering data onto the network, 
and a receiver for receiving data from the network, the 
ring network circulating data around the ring in a time 
interval known as "ring latency", the method compris- 
ing the steps of: 

measuring ring latency at the first station; 
transmitting the measure if ring latency to the second- 
station; 

measuring ring latency at the second station; 

calculating an error signal, at the second station, as 
the difference between measurements of ring la- 
tency made by said first and second stations; and 

modifying the frequency of the timing signal genera- 
tor, at the second station, in a manner that de- 
creases the magnitude of the error signal. 

9. The method of claim 8 wherein the step of measur- 
ing ring latency at a station comprises the following 
steps: 

transmitting a data packet onto the ring network; 

identifying the return of the data packet from the ring 
network; and 

measuring the time interval between steps of trans- 
mitting and receiving said data packet. 

10. The method of claim 9 wherein the station in- 
cludes a counter that counts pulses from the timing 
signal generator, and wherein the time interval mea- 
surement comprises the following steps: 

resetting the counter prior to the time interval mea- 
surement; 

starting the counter when the data packet is transmit- 
ted; and 

stopping the counter when the data packet is re- 
ceived. 

11. A method for establishing frequency synchroniza- 
tion between clocks at two different geographic loca- 
tions, each location having access to a common trans- 
mission media, and each location having apparatus, 
including its clock, for independently measuring signal 
propagation delay over the transmission media, the 
method including the steps of: 

measuring the propagation delay over said transmis- 
sion media at each location: 

communicating the measurement of the propagation 
delay made at one location to the other location; 
and 

changing the frequency of the clock at said other 
location in accordance with the difference between 
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the measurements of propagation delay made at 

each location. 
12. The method of claim 11 wherein the common 
transmission media comprises communication links in- 
terconnecting a plurality of nodes in a ring network, 


12 


and wherein said two different locations correspond to 
two of the nodes. 

13. The method of claim 11 wherein the common 
transmission media comprises fiber-optic cables inter- 
connecting said locations in a ring topology. 
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The invention resides in the field of transferring data and 
other information from multiple asynchronous TDM chan- 
nels across a synchronous interface in digital blocks of a 
preset length. Presently, transfer of data and timing infor- 
mation requires serial transmission of frame payload, quali- 
fied with clock and frame pulse indications. A separate set of 
these signals is required in both directions for each port or 
physical link. An interface of the invention permits data 
transfer in blocks which are uniquely identified for a specific 
port or links and the position of the block within the frame 
(if the transfer is one of framed data). The invention greatly 
reduces the number of required signals, thus enabling reduc- 
tion of pin count requirements and an increase in the number 
of serviced ports. 

13 Claims, 4 Drawing Sheets 


PMD functions 


DS3 


M13 


28 port 

MUX 


T1 


Iramer 


Master 


ORD 


OIS 


OTD 


OCLK 


TC sublayer 
functions 


up to 672 
channel 
HDLC 

Slave 1 


up to 28 

port 
ATM TC 
Sublayer 

Slave 2 


up to 672 
channel 
AAL 1 

Slave 3 


04/01/2004, EAST Version: 1.4.1 


U.S. Patent Mar. 2, 1999 Sheet 1 of 4 5,878,044 


CO « 

1.1 


CD 
CO 


o 

CNJ 


D 
DC 
O 


CO 

O 


q 
21 
Q 

5 


O 

O 


c 
o 

t3 
c 


04/01/2004, EAST Version: 1.4.1 


U.S. Patent 


Mar. 2, 1999 Sheet 2 of 4 


5,878,044 



04/01/2004, EAST Version: 1.4.1 


U.S. Patent Mar. 2, 1999 Sheet 3 of 4 5,878,044 


0 

co co 

IS* 


CO to * 

2§ 







CD 
> 
CO 

° CO 







CD 



° W 


Q 
DC 

o 


CO 

O 


q 

a 
O, 


3 

to 

CO 


O 
O 


to 
c 
o 

o 

c 

3 


CO 


04/01/2004, EAST Version: 1.4.1 


U.S. Patent 


Mar. 2, 1999 


Sheet 4 of 4 


5,878,044 


CD 
>. 

ca to 

2.S 


> 

CO 


O £ -5 J5 


C\J 

CD 

> 

CO 


3 <C0 W 


CVJ _ 

CD _ 
CO C ^ 

2 §^ 


s 

CO 


Q 
DC 
O 


o 
o 


c 
g 

c 


o a> 

Q- E 

00 CO 

CVJ r- ^= 


a) 

to 
ca 


co 


CO 
CO 
Q 


04/01/2004, EAST Version: 1.4.1 


5,878,044 


10 


15 


20 


DATA TRANSFER METHOD AND 
APPARATUS 

FIELD OF THE INVENTION 

The present invention relates generally to interfacing non 
synchronized digital signal streams which exist in different 
sublayers of the physical layer. In particular, it is directed to 
interfacing of digital signal streams by way of a master and 
a slave exchanging blocks of data stream, each comprising 
a predetermined number of bits. 

BACKGROUND OF THE INVENTION 

The current method of transfer of data and timing infor- 
mation between Physical Medium Dependent (PMD) and 
Transmission Convergence (TC) sublayers requires serial 
transmission of frame payload qualified with clock and 
frame pulse indications. A separate set of these signals is 
required in both directions for each port or physical link. 

For example, digital signals are often multiplexed to 
■ produce higher bit rates for higher capacity transmission 
systems, as a means of utilizing the same transmission 
medium economically for many different users. This multi- 
plexing is one of functions that perform such data transfer. 
Different digital signal hierarchies were developed in North 
America, Europe and other parts of the world. In North 
America, DS1.DS2 and DS3 whose bandwidths are respec- 
tively 1.544 Mbit/sec, 6.312 Mbit/sec. and 44.736 Mbit/sec. 
are used. Transmission services which provide these digital 
signals are called Tl, T2 and T3. DSO is a basic TDM signal 
(voice channel signal) of 64 Kbil/sec. Thus, 24 DSO streams 
' are multiplexed to one DSl.four of which arc combined into 
one DS2. Seven DS2 streams are multiplexed to one DS3 
stream. At each multiplexing operation, certain overhead bit 
or bits are added for framing, synchronization and other 
housekeeping functions. In some multiplexing operations, 
certain payload databits are borrowed for housekeeping 
functions as well. 

According to "Four channel DS1 Framcr" Eugene L. 
Parrella et al, IEEE International ASIC Conference and ^ 
Exhibit; 

"Wideband telecommunications services such as T3, 
SONET, and inverse multiplexed Tl services are driv- 
ing more highly integrated multichannel Tl cards. With 
the availability of single chip M13 (T1/T3) multiplex- 45 
ers and multichannel SONET mappers, Tl framers and 
Tl line interface units may become the bottlenecks to 
further reductions in card size, power and cost. A 
multichannel Tl framer offers substantial board area 
reduction, and is highly desirable if competitive in 50 
power consumption and cost to multiple single channel 
framers. 

As an example, a DS0-T3 switching application can be 
considered. A single chip M13 multiplexer is 
employed, multiplexing 28 DSls into a DS3. To con- 
vert DSO's into 28 framed DSl's, 28 DS1 framers are 
required. A multichannel framer can be used effectively 
to lower parts count for the system." 
Therefore, the above application of DS1-DS3 interface 
requires 112 connections between a group of 7 quad DS1 
framers and a M13 multiplexer. 

The present invention permits the transfer of data and 
timing information between the PMD and TC sublayers, a 
predetermined sized block of data at a time. Each data block 
is uniquely identified with an identification number which 
specifies the port or physical link with which it is associated 
and the position of the data block within the frame (if the 
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transfer is one of framed data). Timing information is passed 
between the two sublayers in digital representation form. It 
represents an offset with respect to a reference clock which 
can be board, system, or network wide. 

This invention greatly reduces the number of signals 
required between the two sublayer devices. This makes it 
physically possible to increase the number of ports serviced 
by PMD and TC sublayer devices by greatly reducing their 
pin count requirements. 

In one embodiment, the present invention permits inter- 
facing between digital signals of two different hierarchical 
levels very efficiently. In particular, as an example, in a 
DS1-DS3 interface only 24 lines (instead of 112 connec- 
tions mentioned above) are required, running at a reasonable 
speed of 50 MHz. 

Furthermore, for channelized applications, the necessity 
to demultiplex onto separate asynchronous bit streams only 
to multiplex back to one synchronous stream has been 
eliminated. 

OBJECTS OF THE INVENTION 

It is therefore an object of the invention to provide a 
method and an apparatus for transferring data, timing and 
other control information between two sublayer devices 
efficiently. 

It is another object of the invention to provide a method 
and apparatus for interfacing two sublayer devices which 
require less signals for transferring data and other informa- 
tion. 

SUMMARY OF THE INVENTION 

Briefly stated, according to one aspect, the invention is 
directed to a method of synchronously transferring data and 
other information between a physical medium sublayer and 
a transmission convergence sublayer, each handling a non 
synchronized digital signal stream. The method comprises 
steps of a master bidirectionally handling a non- 
synchronized digital signal stream to/from the physical 
medium sublayer and a slave bidirectionally handling a 
non-synchronized digital signal stream to/from the transmis- 
sion convergence sublayer. The method further includes 
steps of the slave and the master exchanging the data and 
other information in individually identifiable blocks of pre- 
determined lengths during each successive exchange 
sequence of a preset length of time in a time synchronized 
fashion under a clock signal. The method comprises yet 
further steps of, during each exchange sequence, the master 
sending the slave identifiers of blocks of data and other 
information to be exchanged between the slave and the 
master, and in response to the identifiers, the slave exchang- 
ing with the master the blocks of data and other information 
so identified. 

According to another aspect, the invention is directed lo 
an interface for synchronously transferring data and other 
information between a physical medium sublayer and a 
transmission convergence sublayer, each sublayer handling 
a non -synchronized digital signal stream. The interface 
comprises a master for bidirectionally handling a non- 
synchronized digital signal stream to/from the physical 
medium sublayer and a slave for bidirectionally handling a 
non-synchronized digital signal stream to/from a transmis- 
sion convergence sublayer. The interface further includes 
time division multiplexed buses connecting the master and 
the slave for exchanging the data and other information in 
individually identifiable blocks of predetermined lengths 
during each successive exchange sequence of a preset length 
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of time, in a time synchronized fashion under a clock signal, 
in that the identifiable blocks in each successive exchange 
sequence contain data and their identifiers. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a schematic illustration of an interface according 
to one embodiment of the invention; 

FIG. 2 is a timing diagram of signals of one embodiment 
of the invention; 

FIG. 3 is a schematic illustration of an interface according 
to another embodiment of the invention in which more than 
one slave is used; and 

FIG. 4 is a schematic illustration of an interface according 
to yet another embodiment of the invention in which HDLC, 
ATM TC and AAL1 sublayers are shown as examples. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS OF THE 
INVENTION 

Generally speaking, the invention concerns a synchronous 
interface used to transfer multiple bidirectional non- 
synchronized data streams between a PMD and the Physical 
Layer. 

One embodiment of the invention is concerned with 
multiplexing digital signals of different hierarchy levels, in 
which 28 DS1 data streams are interfaced into one DS3 data 
stream. According to DS1 and DS3, the payload data are 
formatted in blocks of an integer multiple of octets. 
Conveniently, therefore, this can be called an Octet Stream 
Interface. This interface defines the exchange of octets of 
data between the two or more devices based on an Octet 
Stream Identifier which uniquely identifies each stream of 
data and related control information. The Octet Stream 
Identifier specifies the port or physical link with which the 
data and control information arc associated as well as the 
byte position of the octet within the frame (if the transfer is 
one of framed data). 

FIG. 1 illustrates schematically one embodiment of the 
invention in which a master and a slave are shown. The 
device which communicates with the PMD is described as 
the Octet Stream Master 20 since it provides the liming and 
sequencing for the transfers based on the availability of 
payload data from the facility and the requirement to provide 
payload data to the facility, e.g. TC sublayer. The device 
providing the TC sublayer is called the Octet Stream Slave 
22 since it simply responds to requests made by the timing 
and sequence master. The direction of flow from the master 
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45 


This is Octet stream Receive Data and contains up to 12 bits 
of data driven from the Octet Stream Master 20 to the Octet 
Stream Slave 22. ORD[ll] is the MSB (most significant bit). 
OTD[9:0] 

This is Octet stream Transmil Data and contains up to 10 bits 
of data driven from the Octet Stream Slave 22 to the Octet 
Stream Master 20. OTD[9] is the MSB. OTD is tri-stated by 
all devices which are not programmed to respond to a 
particular TID value. 
OCLK 

This is Octet stream CLocK and is an octet transfer/ 
synchronization clock for synchronizing transfers on both 
ORD and OTD. 
OIS 

This is Octet stream Interface Sync and is an active high 
signal asserted by the Octet Stream Master during the first 
transfer cycle of an exchange sequence. 

Referring further to FIG. 2, operation and timing are 
described in detail in connection with clock cycles. The two 
unidirectional busses ORD and OTD are TDM busses which 
carry different information depending on the transfer cycle 
in the exchange sequence governed by OIS. An exchange 
sequence is the transfer of one complete set of parameters in 
both the transmit and receive directions. 

During one exchange sequence, payload data blocks RxD 
and TxD identified by the master are exchanged in receive 
and transmit directions. The signalling information RxC and 
TxC so identified by the master are also exchanged in the 
same exchange sequence. The master identifies data and 
signalling information for exchange by sending the slave 
identification numbers in TID and RID. 

According to the embodiment, an Octet Stream Identifi- 
cation number is a number from 0-1023 which is used to 
uniquely identify a specific octet stream. The Octet Stream 
Master applies a base Octet Stream Identifier for each link 
or virtual tributary supported. Consecutive Octet Stream 
Identifiers are then assigned for each channel in a link. If the 
link is uncbannelized, the base octet identification number is 
used for all octets. The Octet Stream Slave device requires 
a programmable internal mapping of octet id's to channel 
numbers to provide a level of indirection. 
Definition of the data content is as follows: 
TID (Transmil IDcntificr) 
This is a 12 bit value sent by the Octet Stream Master device 
which identifies the required TxD and TxC in the next 
exchange sequence. TID[9:0] contains the Octet Stream 
Identifier number, TID[10] is asserted if data is required, and 
TID[11] is asserted if (he octet requested is in the first frame 
of a multiframe, where a multiframe is service specific. An 


to the slave is called the receive direction, and the direction $Q example of multiframe sizes is given in the "DS1/DS3 


opposite thereto is the transmit direction. The transmit and 
receive transfers both use the same transfer clock. If the 
maximum transfer clock is e.g., 50 MHz, the maximum 
transfer rate would be 100 Mbps in each direction. The 
transfer of data in blocks of a predetermined length, e.g. 
blocks of an octet, is via a common transfer clock and is 
synchronized at the block level. 

Referring to FIG. 2, each exchange sequence consists of 
four transfer cycles and achieves transfer of a data block in 
two directions. In this embodiment, the data block is one 
octet long. However, it is, of course, possible to choose any 
reasonable length although, due to welt adapted standards 
which presently exist, certain sizes would be more logical 
than others. 

Referring further to the figure, the following signals are 
required: 
ORD[11:0] 


Example" on pages 8 and 9. Signalling bits will be provided 
by the Octet Stream Slave in the first frame in a multiframe 
if they are valid. 

RID (Receive IDentifier) 
55 This 12 bit value sent by the Octet Stream Master device 
identifies the RxD and RxC in the following two clock 
periods. RID[9:0] contains the Octet Stream Identifier 
number, RID[10] is asserted if valid data is present, and 
RID[11] is asserted if this octet is in the first frame of a 
60 multiframe. Valid signalling bits will be present with each 
octet in the first frame of a multiframe. 

RxD (Receive Data) 
This is an 8-bil payload value which is identified by RID on 
the previous clock cycle. This is valid if RID[10] is asserted. 
65 TxD (Transmit Data) 

This is an 8-bit payload value which is identified by the TID 
in the previous exchange sequence. 
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RxC (Receive Control) 
This value contains control information passed from the 
Octet Stream Master to the Octet Stream Slave. This control 
information is identified by RID in two previous clock 
cycles. The contents of the control field are illustrated in the 
Table below when RxC is set for RCC-011. 


11 10:8 7 6 5 4 3 2 10 
RxControl IS RCC A B OA' D/B" RTS 
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-continued 



DS1 #1 

DS1 Wl 

DS1 #3 

. . DS1 WIS. 

DSO #3 

2 

26 

50 

650 

DSO #24 

23 

47 

71 

671 


io In the transparent mode, the following numbers apply: 


TxC (Transmit Control) 
This value contains control information passed from the 
Octet Stream Slave to the Octet Stream Master. This control 
information is identified by the T1D in the previous block 
exchange sequence. The contents of the control field are 
illustrated in the Table below when TxC is set for TCC«11. 


9:8 7 6 5 4 3 2 1 0 
TxCoauaJ TCC A B OA" D/B" KTS 


RCC-Reccive Control Code. This code indicates what data 
will be present in bits 7-0 as follows: 
000: RXq7:0]-User denned field 
001: RXC[7:4]=Updated Signalling bits 

: RXq3:0>Unused 
010: RXCf7:4]«Unused 

: RXC[3:0}=Updated SRTS value 
011: RXC[7:4]= Updated Signalling bits 

: RXC[3:0>Updated SRTS value 
100-111: RXC[7:0]-Uscr defined field 
TCC=Transmit Control Code. This code indicates what data 
will be present in bits 7-0 as follows: 
00: TXC[7:0>User defined field 
01: TXC[7:4]-Updated Signalling bits 

: TXC{3:0]-Unused 
10: RXq7:4]=Unused 

: RXq3:0>Updated SRTS value 
11: RXq7:4]=Updated Signalling bits 

: RXq3:0)=Updated SRTS value 
ABCD-ABCD Signalling bits 

ABA'B'-AB Signalling bits for two superframes when using 
DS 1 SF format. A'B' are the signalling bits for odd numbered 
superframes and AB is for even numbered superframes. 
LSoLink Status for link corresponding to the RID. This is a 
user defined status field used to pass link information such 
as LOS, OFF, LOF, etc. as required. 
RTS-Rx RTS value. This is CES specific timing 
information, as defined in the 1363.1 specification. This is an 
RTS formatted value used to support SRTS or any other 
clock recovery mechanism. Bit 3-RTS most significant bit. 

In the embodiment where a single DS3 master to a single 
slave is used, the following identification numbers should be 
used in non-transparent mode. 


DSl #1 DSl #2 DSl #3 ... DS1 028 

DS0 ffl 0 24 48 648 

DSO #1 J 25 49 ... 649 



DSl #1 

DSl #2 

DSl #3 

. . DSl «« 

DS0#1 

0 

24 

48 

648 

15 DSO #2 

0 

24 

48 

648 

DSO #3 

0 

24 

48 

648 

DSO #24 

0 

24 

48 

'. '. 648 

It should be noted that the DS1-DS3 embodiments are given 


as example only. It would be apparent to those skilled in the 
art that the invention is equally applicable to other digital 
signal streams, such as El, J2, STS-1, E3, STM-1, etc. 

25 Referring to FIG. 3, another embodiment is schematically 
illustrated in which multiple slaves are provided with one 
master. If multiple masters are required, they must be 
connected in a manner to appear as one Octet Stream Master 
as defined herein. 

30 One other preferred embodiment is schematically illus- 
trated in FIG. 4 in which, as in FIG, 3, multiple slaves are 
provided with one master. The master in this embodiment 
includes a M13 multiplexer for handling DS3 signals in the 
physical medium sublayer. The master includes 28 port Tl 

35 framers which are connected to three slaves by busses and 
other connections. ORD and OTD are time division multi- 
plexed busses. On the transmission convergence sublayer, 
one slave carries up to 672 channel HDLC (high level data 
link control) signals, another up to 28 ATM TC sublayer 

40 signals. The third slave handles up to 672 channel AAL1 
(ATM adaptation layer 1) signals. Of course, other TC 
sublayer signals are possible. 
What is claimed is: 

I. A method of synchronously transferring data and other 
45 information between a physical medium sublayer and a 
transmission convergence sublayer, each handling a non- 
synchronized digital signal stream, comprising steps of: 
a master bidirectionally handling a non-synchronized 
5(J digital signal stream to/from the physical medium 
sublayer; 

a slave bidirectionally handling a non-synchronized digi- 
tal signal stream to/from the transmission convergence 
sublayer; 

55 the slave and the master exchanging the data and other 
information in individually identifiable blocks of pre- 
determined lengths during each successive exchange 
sequence of a preset length of time in a time synchro- 
nized fashion under a clock signal; 
60 the method further comprising steps of: 
during each exchange sequence, 
the master sending the slave identifiers of blocks of 
data and other information to be exchanged between 
the slave and the master; and 
65 in response to the identifiers, the slave exchanging with 
the master the blocks of data and other information 
so identified. 
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2. The method according to claim 1 wherein an exchange a slave for bidirectionally handling a non-synchronized 
sequence comprises a plurality of transfer cycles, and during digital signal stream to/from the transmission conver- 
each transfer cycle, the master sends the slave, respectively, gence sublayer under the control of the master; 

the identifier of the data block to be received from the slave . 

during the next exchange sequence, a data block, other 5 tune dlvlslQn multiplexed buses connecting the master 

information and their identifier. slave for exchanging the data and other infor- 

3. The method according to claim 2 wherein during the mation so identified by the master in individually 
coincidental transfer cycles, the slave sends the master identifiable blocks of predetermined lengths during 
respectively the data block and other information so iden- each successive exchange sequence of a preset length 
tified by the master in the previous exchange sequence. ao of time in a time synchronized fashion under a same 

4. The method according to claim 3 wherein the other clock signal; and 
information exchanged between the master and the slave 

comprises control signals which include a timing signal, 'be identifiable blocks in each successive exchange 

signalling information and/or other user definable informa- sequence containing data and their identifiers. 

tion. is io. The interface according to claim 9, wherein the 

5. The method according to claim 4 wherein the timing identifiable blocks in one successive exchange sequence on 
signal is RTS values. onc time ^vision multiplexed bus contain data and control 

6. The method according to claam 4 wherein the signa hng ^ ^ identifier m tQ ^ idenlifier of data 
information is exchanged by the use of standard stalling * lbe xcoad Ume division muUiple xed 
bits contained m the control signals. 20 . r 

7. The method according to claim 4 wherein there is more bus > n * e following exchange sequence. 

than one slave and the method comprises a further step of: 11. The interface according to claim 10, wherein the 

one of the slaves, in response to the identifiers, exchang- identifiable blocks in one successive exchange sequence on 

ing with the master the blocks of data and other another time division multiplexed bus contain data and 

information so identified. 25 control signals so identified by an identifier on the first time 

8. The method according to claim 7 wherein the non division multiplexed bus in the previous exchange sequence. 

^^Te^f ^ Si !^^ ,reamS ° f DS1 ' E1 ' J2> 12- The interface according to claim U, wherein each 

DS3, E3, STS-1, and STM-1. . . ,. . , , . . . , - . . 

9. An interface for synchronously transferring data and exchan Se sequence is divided mto four transfer cycles and 
other information between a physical medium sublayer and 30 each identifiable block is transferred in each transfer cycle in 
a transmission convergence sublayer, each sublayer han- one direction over one time division multiplexed bus. 
dling a non-synchronized digital signal stream, comprising: 13. The interface according to claim 12, wherein there is 

a master for bidirectionally handling a non-synchronized more than one slave, 
digital signal stream to/from the physical medium 

sublayer; ***** 
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ABSTRACT 


In a local area network composed of transmission lines 
for interconnecting a plurality of subordinate networks 
including synchronous apparatuses and a plurality of 
nodes which connect the subordinate networks to the 
transmission lines, information is transferred using a 
fixed length frame, a clock source which generates an 
independent clock signal and a circuit which generates 
a fixed length frame with the oscillation frequency of 
the clock source as a reference are provided in each 
node so as to adopt an independent clocking system, 
and distribution of a common synchronizing clock re- 
quired for synchronous apparatuses is made by trans- 
mission while embedding transition point information of 
a synchronizing clock in a specific space in the fixed 
length frame. Further, each node generates a fixed 
length transmission frame with an independent clock 
signal, and on the other hand, there are provided in each 
node, from the requirement that information quantity 
applied to a network is made constant, a circuit for 
extracting a received clock, a storage device for storing 
received information temporarily, and an information 
outgoing quantity control circuit in which information 
quantity which is sent out in one frame is increased 
when the information quantity stored in the storage 
device becomes more than a predetermined first refer- 
ence value and information quantity which is sent out in 
one frame is decreased when the information quantity 
stored in the storage device becomes less than a prede- 
termined second reference value. 

18 Claims, 10 Drawing Sheets 
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FIG. 2 
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cation Ser. No. 07/399,901, filed Aug. 29, 1989 which 
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BACKGROUND OF THE INVENTION 

The present invention relates to an architecture of an 
independent clocking local area network (LAN), i.e., a 
local area network (hereinafter abbreviated as a LAN) 
in which each node has an independent clock source of 13 
a clock signal and an information signal is sent out using 
an oscillated clock signal, and more particularly, to an 
architecture of a multimedia LAN in which any data 
transfer error due to jitter accumulation is not gener- 
ated even when the quantity of connected nodes is 20 
increased. 

In a LAN, a plurality of node devices (hereinafter 
simply referred to as nodes) are connected with one 
another with a single transmission line and high speed 
information transmission and switching function are 13 
realized efficiently in a limited service area, and variety 
types of LANs have been put to practical use. A ring 
type LAN having a transmission line in a ring form and 
a bus type LAN using a transmission line in a bus form 
arc typical. In such a LAN construction, a synchronous 30 
system becomes an issue. Ideally, it is preferable that all 
the nodes constituting a LAN are operated with a same 
clock signal (hereinafter simply referred to as a clock). 
In case all the nodes are in operation with the same 
clock, the rate of sending information and the rate of 35 
receiving information are equal to each other. There- 
fore, transmission and reception of information become 
possible without providing a buffer therebetween. As 
such a LAN in which all the nodes are operated with 
the same clock, the standard IEEE 802.5 ('Token 40 
ring") (a document ANSI/IEEE Std 802.5-1985 
ISO/DP 8802/5 "LOCAL AREA NETWORKS 
Token Ring Access Method") is a typical well-known 
example. In respective nodes in a LAN of above-men- 
tioned standard IEEE 802.5, clock components in- 45 
eluded in a received signal from a previous (transmis- 
sion) node are regenerated by means of a phase lock 
loop (PLL), regenerated dock is supplied into a receiv- 
ing node, and further, information is sent out to a next 
node by abovementioned regenerated clock (master- 30 
slave synchronization). The clock for synchronization is 
repeated at respective nodes as described above, makes 
a round in the ring, and the whole system becomes to be 
operated synchronously with a synchronous clock gen- 
erated by a master node. Since the clock is regenerated 55 
and repeated in respective nodes, however, the jitter 
generated at the time of regeneration and transfer of the 
clock is accumulated. Since received data are regener- 
ated by the clock having such jitter, such a problem 
arises that received data are not regenerated correctly 60 
when the jitter becomes larger. The quality of connect* 
able nodes is limited in many cases in point of operation 
by such jitter accumulation. 

In order to avoid jitter accumulation, there is an 
(independent synchronizing) system in which regenera- 65 
tion and transfer of the clock is not performed, each of 
respective nodes has an independent clock source, re- 
spectively, and an information signal is sent out using an 


543 

2 

oscillated clock. For example, this system is described 
in detail in the standard FDD I -I (a document ISO/IEC 
JTC1 SC13 N477; Draft for ISO 9314-1: Fiber Distrib- 
uted Data Interface (FDDI) Token Ring Physical 
Layer Protocol (PHY)). 

In FDD1-I, however, the information signal transmit- 
ted in the LAN is asynchronous information, viz., only 
the information which is not required to be sent periodi- 
cally, and the information is transmitted and received 
with a frame for sending information (hereinafter re- 
ferred to as a frame) (4,500 bytes maximum) as a unit. A 
blank of 8 bytes and more is put between mutual frames, 
and the difference in clock frequencies between nodes is 
absorbed by increasing and decreasing the size of the 
blank portion. Thus, it is possible for respective nodes to 
conduct communication without giving rise to over- 
flow or underflow by regenerating and repeating data 
only. 

The above-mentioned independent clocking system is 
a system which is applicable only to a LAN which 
supports asynchronous data only. Recently, however, 
demand for a high speed LAN called a multimedia 
backbone LAN which is able to transmit and switch not 
only asynchronous data, but also synchronous informa- 
tion is increasing. (Information, voice and data which 
are required to transmit a predetermined quantity peri- 
odically are typical examples. These may be handled as 
asynchronous information, but buffering processing and 
the like are required to guarantee periodicity at trans- 
mit-receive terminals, causing handling to become com- 
plicated.) Such a multimedia backbone LAN accommo- 
dates a low speed, asynchronous-data-dedicated LAN 
such as the standard IEEE 802.3, 802.4 and 602.5 and 
FDDI-I which is a high speed LAN so as to realize 
information transmission and switching function among 
LANs, and also supports information transfer among 
synchronous apparatuses such as a PBX (private branch 
exchange) and a TDM (time division multiplexer) so as 
to realize an integrated private network. Existing syn- 
chronous apparatuses arc designed on the premise that 
these apparatuses are operated with the same synchro- 
nizing clock when they are interconnected. Accord- 
ingly, in a network including such synchronous appara- 
tuses it is required to supply a synchronizing clock to 
synchronous apparatuses from the network through 
nodes. Further, since it is required to transfer informa- 
tion periodically and at a same rate among synchronous 
apparatuses, it is preferable that an information quantity 
applied to respective nodes is made equal in the whole 
system. Thus, it is required to supply a synchronizing 
clock which is common to all nodes. 

As a result, a master-slave synchronization system 
which is easy to be constructed has been heretofore 
employed for synchronization of the multimedia LAN. 
As a document related to such a technique, "A 1.2 Gbps 
optical loop LAN for wideband office communica- 
tions" IEEE Global Telecommunications Conference 
1985, 15-4, may be mentioned. 

In the above-mentioned LAN of master-slave syn- 
chronization system, the synchronizing clock is distrib- 
uted by the fact that the clock generated by a master 
node is regenerated and repeated by respective nodes. 
In this system, since all nodes are operated with a com- 
mon synchronizing clock, it is easy to connect synchro- 
nous apparatuses with one another. Since jitter is accu- 
mulated as described previously, however, there is such 
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a drawback that the number of connectable nodes is 
limited. 

As another system for solving the jitter accumulation 
problem in the multimedia LAN, an independent clock* 
ing system in which respective nodes send out signals to 5 
a transmission line using clocks oscillated in respective 
stations is possible. In the multimedia LAN, however, it 
is necessary to devise how to include the synchronous 
apparatuses, unlike the synchronous-data-dedicated 
LAN. For example, it is being examined to employ an 10 
independent clocking system in the standard FDDI-II 
which is being standardized by the American National 
Standards Institute (ANSI) at present described in de- 
tail in a document: "FDDI Hybrid Ring Control, Draft 
proposed American Standard, Jan. 20, 1989". FIG. 15 15 
shows a construction of a transfer frame (referred to as 
a cycle in FDDMI) adopted in FDDMI. The informa- 
tion is transferred while being embedded in a transfer 
frame of a fixed period. The frame is composed of a 
preamble, a cycle header and an information portion. 20 
The period of the frame is at 125 jis (| KHz). Further, 
the information transmission rate is at 100 Mb/s, but 
information in 4 bits is sent out after converting into 5 
bits (4B/5B code) for the purpose of removing DC 
frequency components on a transmission line and trans- 25 
mission of specific codes (for detection of frame bound- 
ary and control signals). Therefore, the physical trans- 
mission rate is at 123 Mb/s. The number of bits in the 
preamble space is different depending on osculation 
frequency deviation of clocks of respective nodes, but 30 
the number of bits is adjusted so that the cycle period 
becomes 125 us. The master node creates the frame 
period based on an external clock or an oscillation fre- 
quency of the own station. In each node, a synchroniz- 
ing clock is extracted from a received signal using a 35 
PLL, a tank circuit and the like. It becomes possible to 
receive information in a frame by receiving the received 
signal correctly and detecting a synchronous pattern in 
the cycle header using the extracted clock. 

In a proposal in the above-mentioned standard of 40 
FDDMI, the oscillation frequency deviation of each 
node is adjusted by adjusting the length of the preamble 
portion between frames, and periodic data transfer is 
realized by introducing a frame construction. 

It is required for a multimedia LAN to distribute the 45 
same synchronizing clock among synchronous appara- 
tuses through the nodes in order to transmit not only 
asynchronous information, but also synchronous infor- 
mation as described previously. Accordingly, there is a 
problem as a synchronous system in both systems of the 50 
above-mentioned master-slave synchronization system 
and FDDI-II system of independent clocking. That is, 
restriction on the number of nodes due to jitter accumu- 
lation described previously becomes an issue in the 
master-slave synchronization system. 55 

In the FDDI-II system, there are such problems as 
described hereunder. 

A first problem is that the system is weak against a 
transmission error on the transmission line. In a high 
speed LAN, an optical fiber is used for transmission, but 60 
a bit error rate in optical transmission is usually around 
10 -9 . Such a bit error generated at random or in a burst 
form should never be enlarged by the network. In the 
FDDI-II system, a starting point of each frame is recog- 
nized by detecting a specific bit pattern which does not 65 
exist in the information, and there is a possibility that an 
error of one frame portion is generated by the bit error 
at this portion. Further, the length of an outputted 


543 

4 

frame is determined by the length of a received frame in 
each node, and there is also a possibility that a frame 
recognition error of one node extends to a plurality of 
nodes. 

A second problem exists in that a physical transmis- 
sion rate becomes higher than a logical information 
transfer rate. This is caused by the fact that 4-bit infor- 
mation is transmitted after coding into a 5-bit transmis- 
sion code because a specific bit pattern which does not 
appear in the information portion is used for frame 
recognition. In FDDMI, the physical transmission rate 
is set at 125 Mb/s against the information transfer speed 
of 100 Mb/s, and only 80% of the trammission band is 
utilized for actual information transfer. 

A third problem is that frame processing becomes 
complicated because a frame is of a variable length. 

SUMMARY OF THE INVENTION 

Thus, it is an object of the present invention to realize 
an independent local area network and nodes for a local 
area network which have solved above-mentioned 
problems, that is, which have little jitter accumulation 
and are able to distribute the synchronizing clock 
among synchronous apparatuses even when a frame 
having a fixed length is used. 

In order to achieve the above-mentioned object, ac- 
cording to the present invention, there is provided a 
local area network composed of transmission lines for 
interconnecting a plurality of subordinate networks 
including synchronous apparatuses and a plurality of 
nodes which connect the above-mentioned subordinate 
networks to the transmission lines, wherein information 
is transferred using a fixed length frame, a clock source 
generating an independent clock signal in each node 
and means for forming a fixed length frame with the 
oscillation frequency of the clock source as a reference 
are provided whereby to adopt an independent clocking 
system, and distribution of a common synchronizing 
clock required for synchronous apparatuses is made in 
such a manner that transition point information of the 
synchronizing clock is embedded in a specific place in 
the fixed length frame in transmission. The above-men- 
tioned transition point information means that reference 
points such as rising or falling edges of the clock pro- 
vide timewise positional information during the period 
of a transmission frame having a fixed length formed by 
each of the above-mentioned nodes. Since each node 
has an independent clock, transition point information is 
varied with respect to each node because the period of 
a fixed length transmission frame formed by each node 
and the period of a common synchronizing clock are 
independent. 

Further, from such requirements that each node pro- 
duces a fixed length transmission frame with an inde- 
pendent clock signal, and on the other hand, the infor- 
mation quantity applied to a network is made constant, 
there are provided in each node, means of extracting a 
received clock, storage means for storing received in- 
formation temporarily, and information outgoing quan- 
tity control means which increases information quantity 
which is sent out into one frame when the information 
quantity stored in the storage means becomes more than 
a predetermined first reference value and reduces the 
information quantity which is sent out into one frame 
when the information quantity stored in the storage 
means becomes less than a predetermined second refer- 
ence value. 
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The common synchronizing clock is sent out from 
the master node as a preferable embodiment configura- 
tion. Further, NN1 ("Network Node Interface for the 
Synchronous Digital Interface") standards which are 
specified by CCIT (Intemaional Telegraph and Tele* 5 
phone Consultative Committee) standards are applied 
to a physical layer. In particular, a SONET (Synchro- 
nous Optical Network) frame is used as the fixed length 
frame, and the transition point information on the com- 
mon synchronizing clock is transferred by using a sec- 10 
don overhead space of the SONET frame. 

The clock frequency of each node is independent 
(independent clocking), and the rate of transmitted in- 
formation is common in the whole system. Accord- 
ingly, a stuffing function of NN1 standard is used for 15 
absorbing the difference between the node clock fre- 
quency and the information transmission rate in each 
node. The synchronizing clock is distributed by setting 
the overhead portion of NNI standard, viz., the syn- 
chronizing clock period to be distributed at almost the 20 
same frequency as the frame frequency, and by transfer- 
ring transition point information of the synchronizing 
clock. 

According to the present invention, the problem of 
data transmission error caused by clock jitter accumula- 
tion is solved by making the clock frequency in each 
node independent. Furthermore, connectability with 
public networks and appropriation of techniques be- 
come possible by applying NNI standards which are M 
international standards to the physical layer. Further, 
frame synchronization is obtainable by using a fixed 
length frame even if a specific pattern for frame syn- 
chronization is not used. Namely, since a frame syn- 
chronous pattern appears periodically even if the same 35 
pattern as the pattern for frame synchronization is used 
in the information portion, it is possible to recognize a 
starting point of a frame by detecting periodicity. Thus, 
it is possible to make the physical transmission rate and 
the information transmission rate almost equal to each ^ 
other. Further, independent synchronization can be 
realized using the stuffing function of NNI standard. 
Namely, in each node, the difference between a self- 
node clock and the information quantity from a previ- 
ous node is monitored, and stuffing is performed in a 45 
direction of reducing the difference when the difference 
exceeds a specified threshold, whereby making it possi- 
ble to make the information quantity applied to trans- 
mission lines constant while maintaining the clock fre- 
quency in each node independent. The information rate 
applied to the transmission line is specified by the mas- 
ter node. Furthermore, distribution of common syn- 
chronizing clocks required for synchronous apparatuses 
is made possible by transferring transition point infor- 
mation of the synchronizing clock using a control infor- 
mation transfer area (overhead space of NNI standard) 
in a frame. When the distributed synchronizing clock 
frequency is set close to a frame repeat frequency, the 
number of the synchronizing clock transition point in 
one frame is either one of 0, 1 or 2. Therefore, it is 
possible to distribute the synchronizing clock by pre- 
paring a space where information for two transition 
points can be transmitted in the control information 
area. Moreover, overflow and underflow of synchro- 
nous information in nodes are prevented by having the 
information quantity applied in a LAN synchronize 
with the synchronizing clock and supplying the syn* 
chronizing clock to the synchronous apparatuses. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram showing an embodiment of 
an independent clocking LAN according to the present 
invention; 

FIG. 2 is a diagram showing a utilization configura- 
tion of a multimedia LAN; 

FIG. 3 is a frame construction diagram used in a 
LAN of the present invention; 

FIG. 4 is a diagram showing a construction of a sec- 
tion overhead in a frame; 

FIG. 5 is a block diagram showing a construction of 
a node constituting a LAN of the present invention; 

FIG. 6 is a signal construction diagram of a clock 
pointer used in an embodiment of the present invention; 

FIG. 7 is a pattern diagram of synchronizing clocks 
regenerated in an embodiment of the present invention; 

FIG. 8 is a block diagram of a synchronizing clock 
generating circuit in an embodiment of the present in- 
vention; 

FIG. 9 is a block diagram of a clock pointer generat- 
ing circuit in an embodiment of the present invention; 

FIG. 10 is a waveform diagram for explaining a jitter 
generating mechanism in case of synchronizing clock 
transfer; 

FIG. 11 and FIG. 12 both show block diagrams of a 
stuffing portion in a master node in an embodiment of 
the present invention; 

FIG. 13 is a distribution diagram of SOH spaces at 
stuffing buffer input/output; 

FIG. 14 is a block diagram of a stuffing control por- 
tion in a general node; and 

FIG. 15 is a frame format diagram of FDD-I which 
has been heretofore known. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

Both FIG. 1 and FIG. 2 show constructions of multi- 
media LANs in an embodiment of an independent 
clocking LAN according to the present invention. FIG. 
1 shows a part of FIG. 2 in detail for the convenience of 
explanation. In a multimedia LAN shown in FIG. 2, 
nodes 2-1 to 2-13 are connected in a ring form in a 
transmission line 1. The transmission line 1 is an optical 
fiber, and the transmission rate is at 155.52 Mb/s. NNI 
standards of CCITT are adopted for the transmission 
rate of a physical layer and a frame format. Namely, the 
transmission rate is at 155.52 Mb/s. A frame format of 
NNI standard will be explained later with reference to 
FIG. 3. It is possible to connect either one of a synchro- 
nous apparatus and an asynchronous apparatus to a 
multimedia LAN. In FIG. 2, Private Branch Exchanges 
(PBXs) 5-1 and 5-2, remote units (RSU) 6-1, 6-2 and 6-3 
of the PBXs, picture units 7-1 and 7-2, multiplexers 
(MUX) 3-1, 3-2 and 3-3 for communicating with remote 
locations through high speed digital lines are connected 
as synchronous apparatuses. These apparatuses are con- 
nected with one another through channels which guar- 
antee periodical information transfer. On the other 
hand, FDDI-1 4-1, 4-2, 4-3 and 44 which are LANs are 
connected directly to the transmission line 1 through 
nodes 2-3, 2-13, 2-4 and 2-12 of the multimedia LAN as 
asynchronous apparatuses. Work stations (WS), com- 
puters (HOST, CCP) and the like are connected to 
FDDM 4-1, 4-2, 4-3 and 44 directly or through further 
subordinate low speed LANs such as IEEE 802.3, 802.5 
and the like. It is also possible to include them directly 
in the nodes. It is a matter of course to employ a multi- 
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plexer used in common to synchronous and asynchro- and the information transmission rate is adjusted by 
nous apparatuses. In a multimedia LAN, a plurality of varying the size of the information transfer space in one 

synchronous and asynchronous apparatuses are con- frame. Generation of stuffing is informed to a down- 

nected with one another in a high speed, thus realizing stream node by altering the pointer value. The devia- 

information transmission and switching among appara- 5 tion between the frame repeat rate and the information 

tuses. transmission rate is specified by an allowable stuffing 

FIG. 3 shows a construction of a frame transmitted in frequency. Since stuffing can be performed only once in 

the LAN. The frame is the same as a SONET frame four frames according to the NNI standard, the devia- 

which meets the NNI standard and is composed of 270 tion of three bytes is allowable for four frames. With 

rows and 9 columns, and 2,430 bytes (270x 9) are trans- 10 this, it is required that the node clock frequency devia- 

mitted in about every 125 jis (depending on an oscilla- tion of each node falls within ±309 ppm 

tion frequency of a node). Thus, the transmission fre- (=3/(2,430X4)) with respect to the information trans- 

quency is at approximately 155.52 MHz. Among 270 mission rate. Further, in FIG. 4, D1-D12 show data 

rows in the frame, the first 9 rows form a section over- spaces in 12 bytes called data communication channels 

head (hereinafter abbreviated as SOH) space, and are 15 and can be used for control information transmission 

used for control and control information transfer among among the nodes. In the present embodiment, the transi- 

the nodes. The remaining space is used for information tion point information of the synchronizing clock is 

transfer. In order to make an information transfer rate in transmitted by utilizing these spaces. Bytes Bl, B2, CI, 
a multimedia LAN independently of the frequency of El, E2, Fl, Kl, K2, Zl and Z2 are not required for 

the node clock, information is transmitted by using a 20 explaining the present invention. Hence, the explana- 

transmission block of 2,349 (261x9) bytes which is tion thereof is omitted herein. 

called a virtual container 4 (hereinafter abbreviated as Returning now to FIG. 1, information transfer and 

VC-4). The positional relationship between VC-4 and clock distribution in a multimedia LAN will be de- 

the frame is not necessarily fixed, but the positional scribed. In FIG. 1, only the nodes 2-8 to 3-11 in FIG. 2 

relationship between them is varied in accordance with 25 are shown, and other nodes are omitted. Interface por- 
the difference between the node clock and the rate of tions with apparatuses connected to nodes are shown 

information which is transferred along the transmission with respect to the node 2-9 only. The node 2-8 is the 

line. Namely, it is possible to set a starting point of VC-4 master node, and supplies a common synchronizing 

at an arbitrary point (3 byte unit) of the information clock (8 KHz) to other nodes. In FIG. 1, a path 12 

portion in the frame. 30 shows a distribution path of a synchronizing clock, and 

FIG. 4 shows the detail of a SOH space. For example, a path 16 shows a transmission path for information. 

Al and A2 show synchronous patterns, and a starting From a physical point of view, two paths are multi- 
point of a frame is detected by periodical detection of plexed, and transmission is performed using a single 

Al and A2 (the frame period is constant). Further, since transmission path. In the multimedia LAN, an external 

frame synchronous patterns Al and A2 appear periodi- 33 clock 9 (S KHz) supplied from the outside is distributed 

cally, it is able to prevent the occurrence of erroneous among all the nodes 2-8 . . . 2-11 as a common synchro- 

out-of-synchronization due to a bit error generated in nizing clock and is supplied to synchronous terminals 

transmission by deciding a synchronizing error (syn- connected to the nodes from respective nodes. Further, 

chronization protection) with non-detection of the syn- the information transfer rate is determined by output- 

chronous pattern in a plurality of times after frame 40 ting VC-4 synchronously with the external clock 9. 

synchronization is established. An AU pointer (includ- Respective nodes 2*9 .. . 2-11 transfer a common syn- 

ing a stuffing space) in the fourth row of SOH shows chronizing clock information from the master node 2-8 

the starting point of the above-mentioned VC-4. In the to following nodes by means of relay units or repeaters 

information transmission portion in the frame, addresses 11*9 to 11-11. In each node, clock jitter generated at the 

are added in 3 byte unit from the fourth row and the 45 time of clock transfer is reduced and a synchronizing 
tenth column, and the address of the starting point of clock is supplied to a synchronous apparatus 7-2 by 

VC-4 is contained in the AU pointer. Accordingly, it is means of a phase lock loop 13-9 (omitted with respect to 

possible to find out the starting point of VC-4 by look- other nodes). Further, respective nodes include osctfla- 

ing at the AU pointer. When the relative position be- tors 15-8 to 15-11 of the own station, determine frame 

tween the frame and VC-4 is changed (called stuffing), 50 periods with oscillated node clocks and send them to 

the seventh column through the twelfth column of the following nodes. Sniffing buffers 14-8 to 14-11 are used 

fourth row are used. There are two types of stuffing, for absorption of the difference between a receiving 

positive (positive stuffing) and negative (negative stuff- frequency and a transmitting frequency. In case the 

ing), which are used properly so as to compensate for information quantity stored in the buffer b varied by 

the difference in rate in accordance, with the relative 55 ±3 bytes and more from a central value of a buffer 

magnitude between a frame repeat rate determined by capacity, stuffing is executed and adjustment is made so 

the clock frequency of the node and an information that the information quantity applied to the ring be- 

transmission rate. When the frame repeat rate of the comes constant in the whole LAN. The synchronizing 

node is higher, it is required to shift the head of VC-4 in clock information arrives at the node almost periodi- 

a direction in which the head address increases with 60 cally, but does not synchronize completely with the 

respect to the frame. Therefore, adjustment is made by distributed synchronizing clock for a short time by the 

blanking the fourth row and the tenth to twelfth col- influence of stuffing. Therefore, this variation portion is 

umns (positive stuffing). Conversely, when the frame controlled in a delay control circuit 16-9. In the master 

repeat frequency is lower than the information transmis- node 2-8, it is required to provide a function of control- 

sion rate, the head position of VC-4 is altered by trans- 65 ling the delay so that the delay which makes a round in 

ferring the information by using the fourth row and the the ring becomes integer times as long as the frame 

seventh to ninth columns, too (negative stuffing). period so that no information deficiency is produced in 

Namely, the difference between the frame repeat rate transferring operation. Usually, a buffer having a capac- 
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ity of approximately one frame portion is prepared for asynchronous information, both systems are applicable, 

that purpose separately from the stuffing buffer (for The present invention relates principally to a construe- 

example, see JP-B-6 1-44426). This function is described tion of a physical layer. Hence, explanation of an access 

along with a stuffing buffer 14-8 with reference to FIG. system to a ring is omitted herein. The synchronous 

1 for the sake of simplicity. S apparatus is connected to the access control circuit 15 

FIG. 5 shows the construction of the node 2-9 in through a synchronous apparatus interface 30, a syn- 
FIG. 1 in detail. Same numbers are affixed to those parts chronizing buffer 28 for receiving data and a synchro- 
that are the same as components shown in FIG. 1. The nizing buffer 29 for transmitting data (corresponding 
node 2-9 is divided into three regions (A), (B) and (C) collectively to 16-9 shown in FIG. 5). The synchronous 
with one-dot chain lines depending on the type of used 10 apparatus interface 30 terminates a protocol of the syn- 
clock. The region (A) is operated by means of a re- chronous apparatus and converts it into an information 
ceived clock regenerated by an optical receiver 21 from format in the ring. Further, synchronizing buffers 28 
received data. The region (B) is operated by means of a and 29 for receiving and transmitting data are used for 
self-node clock supplied from a clock source 15-9 in- absorbing phase difference and instantaneous frequency 
eluded in the node. The region (C) is operated by means IS difference between a common synchronizing clock and 
of a synchronizing clock supplied from the master node. a node clock. The synchronous apparatus interface 30 
The transmission frequency is at 155.52 MHz, but the belongs to the region (C) and is operated with the corn- 
information is processed in one byte unit in the node. mon syiichrooizing clock. On the other hand, the asyn- 
Therefore, the inside of the node is operated by a clock chronous apparatus is connected through an asynchro- 
at the transmission frequency of i of 155.52 MHz, viz., 20 nous apparatus interface 31, but it is not required to 
19.44 Mhz except a frame synchronous circuit 22. Ac- guarantee periodicity of information transfer. There- 
cordingly, the oscillation frequency of the clock source fore, the asynchronous interface 31 is operated with a 
15-9 of the node is at 1 9.44 Mhz. node clock in a similar manner as the access control unit 

In the region (A), the received optical signal is con- 25. 
verted into an electrical signal in the optical receiver 21, 23 Information is sent out from the node 2-9 in a speci- 
and a transmission clock (155.52 MHz), viz., a clock in Red frame period with the node clock. Therefore, the 
an upstream node is extracted as the received clock. information quantity which VC4 can transfer is differ- 
The extracted received clock is supplied to a frame cnt in general from the information quantity of VC-4 
synchronizing circuit 22, a multiplexing and demulu- which has been input to the node 2-9. A stuffing func- 
plexing/SOH extraction circuit 23 and a clock genera- 30 tion is used in order to absorb the difference in the 
tion circuit 26 in the region (A). A starting point of information quantity. The output of the access control 
frame is detected by the frame synchronization circuit circuit 25 is written into a stuffing buffer 14-9 in byte 
22. The multiplexing and demultiplexing/SOH extrac- unit A stuffing control and frame generating circuit 33 
tion circuit 23 applies serial -parallel conversion to' a generates a frame -which has been explained with refer- 
signal at 155.52 Mb/s so as to convert into a byte unit of 35 ence to FIG. 3 and FIG. 4 and reads the information in 
19.44 MHz and also extracts the SOH portion of the byte unit from the stuffing buffer 14-9 at the time of 
received frame. An elastic buffer 24 in the region (B) is outgoing of VC-4 and sends out the information. With 
used for the purpose of absorbing a phase difference and the stuffing operation, the information quantity in the 
a frequency difference between the received (transmis- stuffing buffer 14-9 is monitored, and execution is per- 
sion) clock and node clock. Among the received frame 40 formed when a predetermined threshold is exceeded, 
information, the portion of VC-4 is written in the elastic For example, when the threshold is set to i of the stuff- 
buffer 24 in byte unit. The information for showing the ing buffer capacity ±3 bytes, negative stuffing is per- 
head of VC-4 is written at the same time. An access formed when the capacity of information accumulated 
control circuit 25 monitors an empty status of the elastic in the stuffing buffer exceeds i of the stuffing buffer 
buffer 24, and causes the VC-4 data to synchronize with 45 capacity +3 bytes, and the information quantity which 
the node clock when VC-4 data are in existence by can be transferred in one frame is increased for adjust- 
reading the data using the node clock in byte unit Since ment. Further, when the capacity of information accu- 
the information quantity of VC-4 is 261/270 of the in- mulated in the stuffing buffer is lower than I of the 
formation quantity in the frame (see FIG. 3), overflow stuffing buffer capacity — 3 bytes, positive stuffing is 
of the elastic buffer 24 is not generated if the node clock 50 performed, and the information quantity which can be 
frequency deviation is controlled within 3.8% (the fre- transferred in one frame is reduced for adjustment. As 
quency deviation is set within 308 ppm practically in described previously, it is required to control the fre- 
order to prevent overflow of the stuffing buffer from quency deviation of the node clock to fall within ±308 
occurring). Further, an access control circuit 25 per- ppm in order not to generate overflow and underflow 
forms relaying of information and information switch- 55 of the stuffing buffer 14-9. In a SOH insertion and multi- 
ing to and from synchronous and asynchronous appara- plex (MUX) circuit 34, SOH information is inserted and 
ruses connected to the nodes. As a method of transfer- information in byte unit is multiplexed so as to obtain 
ring synchronous and asynchronous information, a time information at 155.52 Mb/s. The node clock (19.44 
division type system in which the inside of VC-4 is MHz) is multiplied by light by using a PLL and the like 
divided into two regions, one for transferring synchro- 60 so as to become a clock at 155.52 MHz, which is sup- 
nous information and another for transferring asynchro- plied to a multiplexing circuit 34 and an optical trans- 
nous information, that is, VC-4 is divided into regions mitter 35 (connection omitted in the figure). The infor- 
called slots and each slot is divided, for synchronous mation applied with series conversion is converted into 
information and asynchronous information (slotted an optical signal in an optical transmitter 35 and sent out 
ring) and so forth are adopted. Further, various tech- 65 to an optical fiber which is a transmission line, 
niques (see "The Data Ring Main: An Introduction to Further, the synchronizing clock information existing 
Local Area Network", written by David C. Flint, for in the SOH which is extracted in the SOH extraction 
instance) are well known as an access system to a ring of circuit 23 is sent to the clock generation circuit 26 
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through a signal line 41, and a synchronizing clock is which there is no transition point in the frame. All bits 

generated. The generated synchronizing clock is sup* are set to "1" in both pointers (A) and (B). When a CRC 

plied to the synchronizing buffer 28 for receiving data error is generated in the pointer, received pointer tnfor- 

and the synchronous apparatus interface 30 after jitter is mation is abolished, and a transition point h determined 

suppressed in a PLL 15-9. Further, the generated syn- 5 by counting 2,430 clocks from the last transition point 

chronizing clock is synchronized with the node clock in of the regenerated synchronizing clock. Since it is usu- 

a synchronization circuit 27 (details will be explained ally possible to control the deviation between the node 

with reference to an embodiment shown in FIG. 8). In clock and the synchronizing dock small, it is possible 

a clock pointer generation circuit 32, transition points of with the above to control the influence by a transmis* 

the synchronizing clock synchronized with the node 10 sion error small even if the transmission error is gener- 

clock are counted from the starting point of frame, and ated. 

the counted value is added to the SOH insertion circuit FIG. 8 shows details of the clock generation circuit 
34 as transition point information of the synchronizing 26 and the synchronization circuit 27 of FIG. 5. For the 
clock. The starting point of frame is known with a purpose of simplifying explanation, a check processing 
frame bead signal 36 from the stuffing control/frame 15 circuit of CRC errors is omitted. The frequency of a 
generation circuit 33. In the SOH insertion circuit 34, transmission clock regenerated in the optical receiver 
the transition point information of the synchronizing 21 is divided by eight by the multiplexing and demulti- 
clock is inserted into a data communication channel of plexing/SOH extraction circuit 23, and sent to the clock 
SOH, and is sent out to the following node as synchro- extraction circuit 26 as a regenerated clock (19.44±/3 
nizing clock information. 20 MHz) through the signal line 41 together with extracted 
The distribution method of the synchronizing clock clock pointers (A) and (B). The received two pointers 
will be described in detail hereinafter. FIG. 6 shows a (A) and (B) are used for generating transition point 
transfer format of transition point information of a syn- information of a synchronizing clock in a next frame, 
chronizing clock. A region in S bytes in the whole is Accordingly, subordinate twelve bits of the pointers 
used. For example, transfer is made using D1-D5 of the 25 (A) and (B) are loaded on latches 45 and 50 by means of 
data communication channels of the SONET frame a frame starting signal 36. On the other hand, two 
explained with reference to FIG. 4. Taking a case twelve bit counters 43 and 48 are reset by means of the 
where there are two transition points of the clock in one frame starting signal 36 and start counting up. When the 
frame into consideration, two clock pointers showing value of the counter 43 (or 48) and the value of the latch 
the transition point of the synchronizing clock are trans- 30 45 (or 50) are in agreement with each other, an output of 
ferred. The last one byte is a CRC (cyclic redundancy a comparator 44 (or 49) reaches *'H", and sets a set/re- 
check) code for error check. FIG. 7 shows the relation- set type flip-flop 47 (or 52). The flip-flops 47 and 52 are 
ship between the transfer format and the regenerated reset by signals which are delayed in delay elements 46 
clock. The synchronizing clock (8 KHz) is sampled and 51, respectively. Therefore, when the counter value 
with the node clock (1 9.44 ±0MHz) of a previous node, 35 agrees with the pointer value, a pulse is generated at 
and the transition point information is transferred in the that point. Since the maximal counter value is 2,430, the 
format shown in FIG. 6. Since the period (125 ns±a) of pointer and the counter are not in agreement with each 
the common synchronizing clock and the frame period other when all the bits of the pointer are at "1", and no 
generated by a previous node are different from each clock is generated. Since the output of the flip-flop 47 
other, there are a case in which the transition point is 40 shows the transition point by the pointer (A), and the 
nonexistent (case (111) shown in FIG. 7), a case in which output of the flip-flop 52 shows the transition point by 
there is one transition point (case (I)) and a case in the pointer (B), it is possible to regenerate a synchroniz- 
which there are two transition points (case (II)) in one ing clock by obtaining an OR of two outputs with an 
frame. In each node, the frequency of a transmission OR gate 53. Since the output of the clock generation 
clock (155.52 MHz) regenerated from a received optical 45 circuit 26 is in synchronism with the regenerated node 
signal is divided by eight so as to regenerate a node. clock of a previous node, it is impossible to use the 
clock of a previous node, and a common syenhronizing output as it is in a clock pointer generation circuit 32 
clock is regenerated using the node clock and the transi- which is operated with a self-node clock. Therefore, the 
tion point information of the synchronizing clock. 2,430 output is synchronized with the self-node clock in a 
pes. of node clocks are included in one frame, which are 50 clock synchronization circuit 27. The synchronization 
counted from the starting point of frame, and the circuit 27 is composed of cascase connection in two 
pointer (FIG. 6) shows the clock in which the synchro- stages of two edge-trigger flip-flops 54 and 55. The 
nizing clock has varied. Thus, it is possible to express self-node clock is supplied to two flip-flops 54 and 55. 
the pointers (A) and (B) in 12 bits, but 2 bytes are used Since the input of the flip-flop 54 and the self-node 
for each pointer in order to delimit information in byte 55 clock arc asynchronous with each other, the output 
unit. The pointer (A) shows the first synchronizing becomes unstable sometimes. However, the output is 
clock transition point in the frame, and the pointer (B) synchronized with the self-node clock by taking the 
shows the second transition point. When there is not output of the flip-flop 54 into the flip-flop 55 when the 
relevant transition point, all bits are set to "1". In the unstable state is dissolved. An output 38 of the clock 
case (I) of FIG. 7, there is one transition point in the 60 synchronization circuit 27 is added to the clock pointer 
frame, which is shown with the pointer (A). In the generation circuit 32. 

receiving node, the regenerated node clocks are FIG. 9 shows the detail of the clock pointer genera- 
counted from the starting point of the received frame, tion circuit 32. The number of self-node clocks from the 
and a common synchronizing clock is regenerated by starting point of frame in a self-node to the transition 
generating a pulse when Nl (value of pointer (A)) pes. 65 point of the synchronized synchronizing clock 38 is 
are counted. In the case (II), there are two transition counted by a 12-bit counter 64. A 2-bit counter 63 
points, and pulses are generated each time when Nl and counts the number of synchronizing clock transition 
N2 clocks are counted. The case (III) shows a case in points in one frame. Both counters 64 and 63 are reset 
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by a frame starting signal 36. Further, all the bits of varying parameters of the PLL and by varying the 

latches 62 and 66 are set to by the frame starting frequency of sampling the synchronizing clock. Since 

signal 36. With this, "1" is output for all in case there is sampling is made at 19.44 MHz in the embodiment, the 

no clock transition point in the frame. The bit bO output jitter generated during relaying operation was 50 ns at 

of the counter 63 shows that the transition point of the 5 the maximum, but it is possible to reduce the jitter quan- 

clock is either the first (b0= 1) or the second (b0=0) in tity by increasing the sampling frequency because the 

the frame. The output of an AND gate 61 is varied at generated jitter is reduced in inverse proportion to the 

the first transition point, and the value of the counter 64 sampling frequency. 

at that time is taken into the latch 62. Further, the out- In the next place, stuffing for absorbing the difference 
put of an AND circuit 65 is varied at the second transi- 10 between the node dock frequency and the information 
tion point and the counter value at the transition point is transfer rate in each node will be described in detail. In 
taken into the latch 66. The output of the counter 63 is the embodiment shown in FIG. 1, the information trans- 
varied at the transition point of the synchronizing clock, fer rate is specified by an external clock 9. Namely, 
and a delay circuit 71 is inserted in order to prevent the when synchronization between the starting point of 
pulse width of the outputs of the AND gates 61 and 63 IS VC-4 and the external clock 9 is off, stuffing is per- 
from narrowing. The number of synchronizing clock formed to have both of them in agreement with each 
transition points and the positions of the transition other in order to cause the starting position of VC-4 
points in one frame are found when looking at outputs generated by the master node 2-8 to synchronize with 
69 and 70 of the counter 63 and outputs 67 and 68 of the the external clock 9. On the other hand, stuffing is per- 
leches 62 and 66. Thus, a clock pointer, viz., transition 20 formed in general nodes 2-9 to 2-11 except the master 
point information of the synchronizing clock which is node 2-8 in order to send out the information quantity 
sent to a node on the next stage is generated using the which is sent from a previous node without excess and 
above findings. deficiency with the self-node clock. Accordingly, stuff- 
Next, clock jitter in the above-mentioned synchroniz- ing algorithms are different between the master node 
ing clock distribution will be described. The synchro- 25 2-8 and general nodes 2-9 to 2-11. 
nizing clock is repeated in succession in respective FIG. 11 shows a composition for realizing stuffing in 
nodes, but jitter is generated when the synchronization the master node 2-8. Namely, a construction 33' of a 
circuit 27 of each node performs synchronization. FIG. circuit corresponding to a stuffing control/frame gener- 
10 shows a generating mechanism of the jitter. As it is ation circuit 33 of the general node 2-9 is shown. Stuff- 
understood from the figure, the transition point of the 30 ing in the master node is executed so that the external 
generated synchronizing clock (input of F/F (54)) and clock and the starting point of VC-4 generated by the 
the transition point . of the synchronizing clock after master node are compared with each other and both 
synchronization (output of F/F (55)) arc shifted from phases fall within a fixed value. With this, it is possible 
each other by one clock period + Ax. Since node clock to make the rate of information outputted from the 
frequencies of respective nodes are different from one 35 master node agree with the external clock. In FIG. 11, 
another, Ax is varied time- wise so as to form a jitter. a clock input 17 is an output obtained by applying the 
Since Ax is varied from 0 to 50 ns (1/19.44 MHz) at the external clock 9 of FIG. 1 to the PLL 10 and reducing 
maximum, the maximum value of the jitter reaches 50 the jitter. This clock input 17 is synchronized with a 
nsX number of nodes in the worst case. However, this node clock from a clock source 15-8 in a synchroniza- 
jitter is suppressed by the PLL. Since the jitter attenua- 40 tion circuit 82 (the construction thereof is the same as 27 
tion quantity of the PLL is in proportion to the jitter shown in FIG. 8), and is compared with a starting point 
frequency in general, it is possible to attenuate high signal 87 of VC-4 which is generated by a frame genera- 
frequency jitter to such a level that the jitter causes no tion control circuit 86. A counter 83 is reset by the 
problem in the PLL. Accordingly, the jitter in a low synchronized external clock and counted up with the 
frequency becomes an issue. In order to evaluate the 45 node clock. Therefore, the number of node clocks from 
jitter quantity in case the number of connected nodes is the startup of the synchronized external clock is 
increased, a case of the number of nodes at 128 is evalu- counted. It is possible to know the phase difference 
ated for instance. Now, a case in which the jitters of 50 between the external clock and the starting point of 
ns at the maximum in respective nodes are added to VC-4 by loading the value of the counter 83 onto a latch 
become jitter of a single frequency at the 128th node is 50 84 by a VC-4 starting point signal 87 which is outputted 
considered as the worst case. The maximum amplitude from the frame generation control circuit 86. Since the 
of the jitter becomes 25 ns X 127 transfers *» 3.3 >is. Since phase difference is distributed from 0 to 2,429 (number 
the jitter is usually specified at 10 Hz and above (the of node clocks in one frame - 1), it is possible to control 
jitter at less than 10 Hz is called "wander"), the jitter at so that the starting point of VC-4 falls within three 
10 Hz is considered. A case in which the jitter added in 55 clocks from the synchronized external clock by per- 
the worst case shows a sinusoidal wave at 10 Hz is forming negative stuffing when the phase difference is, 
considered. In this case, all the jitter power is concen- for example, a value from 4 to 1,215 and positive stuff- 
trated to 10 Hz. When it is assumed that the jitter atten- ing is a value from \2 1 5 to 2,426. Decision of the phase 
uation quantity at 10 Hz in the PLL is 30 dB (a value difference is made by means of a decision circuit 85, and 
which is able to be realized easily with existing tech- 60 the result is sent to the frame generation control circuit 
niques by using a voltage controlled crystal oscillator), 86, thus executing stuffing. 

the jitter of the PLL output becomes approximately 100 FIG. 12 shows a construction of another embodiment 

ns which is an allowable value (for example, a user/net- of a stuffing control portion in the master node. In the 

work interface at 1.5 Mb/s is specified in TTC standard present embodiment, it is controlled so that the differ- 

JT-1431, but 3.2 (is is specified in the frequency range 65 ence between the number of bytes of VC-4 which arc 

from 10 Hz to 120 Hz as a jitter quantity to be allowed actually sent out during one period of the synchronized 

by the terminal). It is also possible to control the jitter external clock and the number of bytes (261 X 9—2,349) 

quantity by altering the jitter attenuation quantity by which are to be sent originally becomes a fixed value 
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and less. In FIG. 12. those pans that are the same as 
FIG. 11 are affixed with same numbers, and explanation 
thereof is omitted. Since the counter 83 is reset by a 
signal obtained by synchronizing the external clock 17, 
it is possible to count the VC-4 output signal supplied by 5 
the frame generation control circuit 86 for one period 
portion of a signal obtained by synchronizing the exter- 
nal clock. At the time when counting is terminated, the 
difference between 23 and 49 is obtained by means of a 
subtraction circuit 89 and accumulated with an accumu- 10 
lator 88. It is decided by the decision circuit 85 that the 
accumulated value exceeds +3 bytes, and the result is 
sent to the frame generation circuit 86, thereby to con- 
trol stuffing in the master node. 

In case the external clock 9 cannot be utilized, the 15 
frequency of the clock output from the clock source 
15-8 of the master node 2-8 is demultiplied to generate a 
signal of 8 KHz, which is used as a synchronizing clock 
source. In this case, stuffing is not generated because the 
node clock and the synchronizing clock are in synchro- 20 
nism with each other in the master node. 

Next, a stuffing control portion in a general node will 
be explained in detail. FIG. 13 shows information which 
is inputted and outputted in and from the stuffing buffer 
14-9 shown in FIG. 5. Hatched portions in the figure 25 
show SOH spaces. The frame structure is shown in 
FIG. 3, but the frame is transmitted from left to right 
and from top to bottom successively. Therefore, the 
SOH space including 9 bytes appears periodically as 
shown in FIG. 13. Further, since frame periods and 30 
starting points of frame of reception and transmission 
are independent, respectively, the SOH spaces of trans* 
mission and reception are not in synchronism with each 
other as shown in FIG. 13, and the phase difference is 
varied time-wise. Accordingly, such a problem arises 35 
that when the node determines whether or not stuffing 
is to be performed based on an information quantity in 
the stuffing buffer 14-9. For example, since the input is 
the SOH space in 9 bytes during the period from a point 
a to a point b in FIG. 13, information is not written in 40 
the stuffing buffer, but the output is read out of the 
stuffing buffer 14-9 because the output is an information 
space. Thus, the information quantity in the stuffing 
buffer at the point b is reduced by 9 bytes as compared 
with the point a. In such 8 manner, the information 45 
quantity in the stuffing buffer 14-9 depends on the time 
of observation and is varied by ±9 bytes. In order to 
avoid such a problem, a system is adopted, in which the 
position in the buffer where information has been read is 
stored when information in one frame is read out of the 50 
stuffing buffer 14-9 in byte unit and it is decided 
whether stuffing is executed or not by a mean value of 
one frame portion of the information quantity in the 
stuffing buffer 14-9. According to this system, stuffing is 
performed correctly because reduction of the informs- 55 
tion quantity in the stuffing buffer due to writing in the 
SOH space and increase of the information quantity in 
the stuffing buffer due to writing of SOH are offset each 
other by averaging even under a status such as shown in 
FIG. 13. 60 

FIG. 14 shows a construction of an embodiment of 
the stuffing buffer 14-9 and the stuffing control and 
frame generation circuit 33 shown in FIG. 5 for explain- 
ing the above-mentioned algorithm. The stuffing buffer 
14-9 is composed of a buffer memory 93 which stores 65 
information in byte unit and counters 94 and 95 which 
control write and read addresses, respectively. The 
counter 94 is counted op by a write signal 92 every time 
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information is written from an access control circuit 25 
through a line 39, and the counter 95 is counted by a 
read signal 104 every time information is read by the 
stuffing control and frame generation circuit 33. The 
counter value is reset when it reaches the maximum 
capacity of the buffer 93. Thus, it is possible to know the 
information quantity in the buffer 93 by obtaining the 
difference between both counters 94 and 95 by a sub- 
traction circuit 96. The results thereof are accumulated 
for one frame portion using an adder 97 and a latch 98. 
In order to obtain accumulation for one frame portion, 
the latch 98 is reset by a frame starting signal 101, and 
supplies a clock 105 only when information (excluding 
the SOH space) is read out A decision circuit 99 de- 
cides whether stuffing is to be performed in a next frame 
from the accumulated value of the information quantity 
in the buffer 93 and the number of transfer bytes in one 
frame at the point when accumulation of one frame 
portion is completed. Namdy, the accumulated value 
determines stuffing or no stuffing with the number of 
VC-4 transfer bytes x (the maximum capacity of the 
buffer 93/2 ±3 bytes) as a boundary. The number of 
transfer bytes in one frame has only three types, that is, 
2,346 bytes (positive staffing), 2,349 bytes (no stuffing) 
and 2,352 bytes (negative stuffing) depending on the 
existence of sniffing in that frame, and is informed to the 
decision circuit 99 from a frame generation, control 
circuit 102 through a signal line 106. The result of deci- 
sion is transferred to a frame generation control circuit 
102 through a signal line 100, thus executing stuffing. 

An embodiment of the present invention has been 
described above, but it is clear that the present inven- 
tion is not limited to the above-mentioned embodiment 
Explanation has been made so far with respect to a 
single frame. A plurality of frames are applied practi- 
cally with time division multiplexing for transmission in 
many cases, but a case of transmission applied with time 
division multiplexing is included in the present inven- 
tion as a matter of course. 

For example, when four frames are transmitted with 
time division multiplexing being applied thereto (infor- 
mation transmission rate is 155.52x4 Mbps), it may be 
arranged so as to perform stuffing for four frames at the 
same time instead of performing stuffing for each frame 
(155.52 Mbps). 

According to the present invention, it is possible to 
compose a multimedia LAN which has no jitter accu- 
mulation to a transmission clock and is able to control 
the jitter of the synchronizing clock to such a level that 
has no problem, Furthermore, it is possible to use a fixed 
length frame which is standardized internationally, and 
to realize a LAN which is durable against transmission 
errors and in which the physical transmission rate and 
the logical transmission are equal to each other. 

We claim: 

1. An independent clocking local area network in 
which a plurality of nodes are connected through trans- 
mission lines, wherein each of said nodes comprises: 

means for extracting a received clock signal; 

a clock source which generates an independent node 
clock signal; 

frame generating means for generating a fixed length 
frame with an oscillation frequency of said inde- 
pendent node clock signal as a reference; 

means for regenerating a synchronizing clock signal 
which has been sent out by an upstream node using 
transition point information provided in a specific 
space of a received frame; 
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synchronization means for synchronizing the regen- 
erated synchronizing clock signal with the inde- 
pendent node clock signal; and 

means for detecting a transition point of the synchro- 
nizing clock signal synchronized by said synchro- 5 
nization means and setting said detected transition 
point in a specific space of the fixed length frame 
generated by said frame generation means as the 
transition point information of the synchronized 
synchronizing clock signal. 10 

2. An independent clocking local area network ac- 
cording to claim 1, wherein said means for detecting 
and setting the transition point in each of said nodes 
farther comprises: 

means for counting the time from a position of said 15 
generated fixed length frame to the transition point 
of the synchronizing clock signal synchronized 
with said independent node clock signal; and 

means for inserting a value obtained as a result of 
counting into said specific space of said generated 20 
fixed length frame. 

3. An independent clocking local area network ac- 
cording to claim 1, wherein said generated fixed length 
frame is composed of a SONET frame, and the specific 
space in said generated fixed length frame which trans- 25 
fers the transition point information of said synchroniz- 
ing clock signal is a data communication channel. 

4. A local area network according to claim 1, wherein 
at least one node of said plurality of nodes is a master 
node for feeding said synchronizing clock signal as a 50 
master synchronizing clock signal to remaining nodes, 
and said master node comprises means for using an 
externa] clock signal or said independent node clock 
signal as said master synchronizing clock signal, and 
setting a transition point of said master synchronizing 33 
clock signal in a specific space of said generated fixed 
length frame and means for adjusting an information 
quantity carried in said generated fixed length frame so 
that an information quantity specified by said master 
synchronizing clock signal is transferred to the next 40 
node. 

5. A node for a local area network, comprising: 
receiving means for receiving information of a fixed 

length frame; 

means for extracting a received clock signal; 45 
a clock source which generates an independent node 
clock signal; 

frame generation means for generating a fixed length 
frame with an oscillation frequency of said inde- 
pendent node clock signal as a reference; 50 

regenerating means for reading transition point infor- 
mation of a synchronizing clock signal out of a 
specific space of said fixed length frame received 
by said receiving means and for regenerating said 
synchronizing clock signal which has been sent 55 
from an upstream node using said received clock 
signal; 

synchronization means for synchronizing said regen- 
erated synchronizing clock signal with said inde- 
pendent node clock signal generated by said clock 60 
source; 

means for detecting a transition point of said synchro- 
nized regenerated synchronizing clock signal in 
said received frame; and 

means for inserting the transition point of said syn- 65 
chronized regenerated synchronizing clock signal 
into a specific space in the generated fixed length 
frame. 


6. An independent clocking local area network com- 
posed of a plurality of nodes and transmission lines 
connecting said plurality of nodes, wherein each of said 
nodes comprises: 

means for extracting a received clock signal; 

a clock source which generates an independent node 
clock signal; . 

storage means for storing received information tem- 
porarily; 

means for generating a fixed length frame with an 
oscillation frequency of said independent node 
clock signal as a reference; and 

information outgoing quantity control means for in- 
creasing information quantity which is carried by 
- each frame when information quantity stored in 
said storage means becomes more than a predeter- 
mined first reference value, and decreasing infor- 
mation quantity which is carried by each frame 
when information quantity stored in said storage 
means becomes less than a predetermined second 
reference value; 

wherein each of said nodes comprise means for trans- 
ferring a synchronizing dock signal to be fed to 
synchronous terminals by setting a transition point 
of said synchronizing clock signal in a specific 
space of said generated fixed length frame. 

7. An independent clocking local area network ac- 
cording to claim 6, wherein said transfer means includes 
means for counting the time from a position of said 
generated fixed length frame to a transition point of said 
synchronizing clock signal, and means for inserting a 
value of the counting result into said specific space. 

8. An independent clocking local area network ac- 
cording to claim 6, wherein said plurality of nodes are 
connected in a ring form. 

9. An independent clocking local area network ac- 
cording to claim 6, wherein said generated fixed length 
frame is composed of a SONET frame, and said specific 
space in said generated fixed length frame which trans- 
fers the transition point of said synchronizing clock 
signal is a data communication channel. 

10. A local area network according to claim 6, 
wherein at least one node of said plurality of nodes is a 
master node for feeding said synchronizing clock signal 
as a master synchronizing clock signal to remaining 
nodes, and said master node comprises means for using 
an external clock signal or said independent node clock 
signal as the master synchronizing clock signal and 
setting a transition point of said master synchronizing 
clock signal in said specific space of said generated fixed 
length frame and means for adjusting an information 
quantity carried in said generated fixed length frame so 
that an information quantity specified by said master 
synchronizing clock signal is transferred to the next 
node. 

11. An independent clocking local area network com- 
posed of a plurality of nodes and transmission lines 
connecting said plurality of nodes, wherein each of said 
nodes comprises: 

means for extracting a received clock signal; 
a clock source which generates an independent node 
clock signal; 

storage means for storing received information tem- 
porarily; 

means for generating a fixed length frame with an 
oscillation frequency of said independent node 
clock signal as a reference; and 
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information outgoing quantity control means for in- master node for feeding said synchronizing clock signal 

creasing information quantity which is carried by as a master synchronizing clock signal to remaining 

each frame when information quantity stored in nodes, and said master node comprises means for using 

said storage means becomes more than a predeter- an external clock signal or said independent node clock 

mined first reference value, and decreasing infor- 5 signal as the master synchronizing clock signal and 

mation quantity which is carried by each frame setting a transition point of said master synchronizing 

when information quantity stored in said storage dock signal j n said specific space of said generated fixed 

means becomes less than a predetermined second length frame and means for adjusting an information 

reference value; quantity carried in said generated fixed length frame so 

wherein said information outgoing quantity control 10 tnat an information quantity specified by said master 

means includes a means for increasing or decrees- synchronizing clock signal is transferred to the next 

ing the information quantity in each single frame in code. 

accordance with a result of accumulating informs- 16 ^ independent clocking local area network ac- 
tion indicating the quantity of information stored in cording t0 claim u , wherein said generated fixed length 
said storage means over the ; duration of at least one 15 frame is composed of a SONET frame, and said specific 
frame or a result of averaging informauon mdicat- fa ^ gencretcd feed length frame which trans- 
tng the quantity of information stored in said stor- fefs ^ ^ of synchronizing clock 
age means over the duration of at least one frame; signal ^ a data communication channel. 
wherein each of said nodes comprise means for trans- 1? a local area network according to claim 9, 
ferrmg a synchronizing clock signal to be fed to 20 ^ ^ ^ nodc of ^ of Dodes fc 
synchronous terminals by settmg a Uansmon point node for fcedin said syn Z 0J ^ g clock ^ 
of said synchronizing clock signal in a specific v r . , , . ,° 
space generated fixed length frame. M ? ^ T f amm& 

12. An independent clocking local area network ac- Dodes ; ™ d node comprises mcans /f f"? 
cording to claim 11, wherein said generated fixed length 25 ™ cxtrrrial dock signal or said ^dependent node clock 
frame is composed of a SONET frame, and said specific thc T** ?ynchronizmg clock signal and 
space in saidVenerated fixed length frame whichVans- ■f 11 ?* * of said master synchronizing 
fers the transition point of said synchronizing clock f ^ignal in said specific space of said generated fixed 
signal is a data communication channel ^ frame » d mc f s for justing an informanon 

13. A local area network according to claim 11, 30 quantity earned m said generated fixed length frame so 
wherein at least one node of said plurality of nodes is a « information quantity specified by said master 
master node for feeding said synchronizing clock signal synchromzmg clock signal is transferred to the next 
as a master synchronizing clock signal to remaining node. 

nodes, and said master node comprises means for using 18 A local area network according to claim 6, 

an external clock signal or said independent node clock 35 wherein at least one node of said plurality of nodes is a 

signal as the master synchronizing clock signal and m a ster node for feeding said synchronizing clock signal 

setting a transition point of said master synchronizing 35 a master synchronizing clock signal to remaining 

clock signal in said specific space of said generated fixed nodes, and said master node comprises means for using 

length frame and means for adjusting an information 0X1 external clock signal or said independent node clock 

quantity carried in said generated fixed length frame so 40 signal as the master synchronizing clock signal and 

that an information quantity specified by said master setting a transition point of said master synchronizing 

synchronizing clock signal is transferred to thc next clock signal in said specific space of said generated fixed 

node. length frame and means for adjusting an information 

14. An independent clocking local area network ac- quantity carried in said generated fixed length frame so 
cording to claim 11, wherein said plurality of nodes are 45 that an information quantity specified by said roaster 
connected in a ring form. synchronizing clock signal is transferred to thc next 

15. A local area network according to claim 8, node, 
wherein at least one node of said plurality of nodes is a 

50 


55 


60 


65 


04/01/2004, EAST Version: 1.4.1 


i mi Qmu di iiio inn mi ii mi mil iioi inn mm m mn hb 

US0056Q2841A 

United States Patent [19] [li] Patent Number: 5,602,841 

Lebizay et al. [45] Date of Patent: Feb. 11, 1997 


[54] EFFICIENT POINT-TO-POINT AND 

MULTI-POINT ROUTING MECHANISM FOR 
PROGRAMMABLE PACKET SWITCHING 
NODES IN HIGH SPEED DATA 
TRANSMISSION NETWORKS 

[75] Inventors: Gerald Lebizay, Vence; Jean M. 

Munier, Cagnes sur Men Andre 
Pauporte, La Colle sur Loup; Victor 
Spagnol, Cagnes sur Mer, all of France 

[73] Assignee: International Business Machines 
Corporation, Arraonk, N.Y. 

[21] Appl. No.: 404,800 

[22] Filed: Mar. IS, 1995 

[30] Foreign Application Priority Data 

Apr. 14, 1994 [EP] European Pal. Off. 94480030 

[51] IntCI. 6 H04L 12/56 

[52] U.S. CI ~.. 370/413 

[58] Field of Search » 370/58.2, 60, 60. 1 , 

370/61,79,94,1.94.2 

[56] References Cited 

U.S. PATENT DOCUMENTS 

5.365,519 11/1994 Kozaki et al 370760 

5,393,280 2/1995 Zheng - - 370/60 

FOREIGN PATENT DOCUMENTS 

0575281 12/1993 European P&L Off. H04L 12/18 

0579567 1/1994 European Pat. Off. H04L 12/56 

0632625 1/1995 European Pal. Off. H04L 29/06 

9102420 2/1991 W1PO H04L 12/56 

OTHER PUBLICATIONS 

Globecom *91, vol. 1, 12-92 NY pp. 104-110. 

Int. Journal of Digital and Analog Communication Systems, 

vol. 5, 1992 pp. 29-37. 


Proceedings of the IEEE '92 Custom Integrated Circuits 
Conf., 5-29, NY, pp. 14.4.1-14A4. 

Primary Examiner— Douglas W. Olms 

Assistant Examiner— Russell W. Blum 

Attorney, Agent, or Firm— Stephen T. Keohane; John J. 

Timar 


[57] 


ABSTRACT 


The present invention relates to an efficient point-to-point 
and multi-points routing system and method for program- 
mable data communication adapters in packet switching 
nodes of high speed networks. The general principles of this 
efficiency are the following: 
First, data packets are never copied, only packet pointers 
are copied for each destination: Space in Buffer 
Memory is saved, the number of instructions is signifi- 
cantly reduced improving the packet throughput (num- 
ber of packets per seconds that the adapter is able to 
transmit), and the routing is independent of the packets 
length. 

Second, no overhead is generated by the multi-points 
mechanism in the real time procedures: the underrun/ 
overrun problems on the ouputs are reduced and the 
efficiency of the adapter in term data throughput (bits 
per second) is significantly improved. 

Third, each output is processed independently by means 
of interrupts: lines are managed in real time and lines 
of different speed or protocol can be supported in 
parallel. 

Fourth, the release of the resources is entirely realized on 
a non priority mode. 
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EFFICIENT POINT-TO-POINT AND 
MULTI-POINT ROUTING MECHANISM FOR 
PROGRAMMABLE PACKET SWITCHING 
NODES IN HIGH SPEED DATA 
TRANSMISSION NETWORKS 5 

TECHNICAL FIELD 

The present invention relates to an efficient point-to-point 
and multi-points routing system and method for program- ]0 
mable data communication adapters in packet switching 
nodes of high speed networks. 

BACKGROUND ART 

The telecommunication environment is in full evolution 15 
and has changed considerably this recent years. The princi- 
pal reason has been the spectacular progress realized in the 
communication technology: 

the maturing of fiber optical transmission. High speed 
rates can now be sustained with very low bit error rates. 20 

the universal use of digital technologies within private 
and public telecommunications networks. 

In relation with these new emerging technologies, the 
offer of the telecommunication companies, public or private, ^ 
are evolving: 

The emergence of high speed transmissions entails an 
explosion in the high bandwidth connectivity. 

the increase of the communication capacity generates 
more attractive tariffs, 30 

A higher flexibility is offered to the users to manage their 
growth through a wide range of connectivity options, an 
efficient bandwidth management and the support of new 
media. 

Once sampled and digitally encoded, voice, video and 
image derived data can be merged with pure data for a 
common and transparent transport- 
Abundant, cheap communications means that many 
potential applications that where not possible before because ^ 
of cost are now becoming attractive. In this environment, 
three generic requirements are expressed by the users; 
Doing old applications better, 
Optimizing communication networks, 
Doing new applications. 45 

High Performance Networks 

In a first step, Tl backbone networks were primarily 
deployed with TDM (Time Division Multiplexing) technol- 50 
ogy to achieve cost savings through line aggregation. These 
systems easily supported the fixed bandwidth requirements 
of host/terminal computing and 64 Kbps PCM (Pulse Code . 
Modulation) voice traffic. 

The data transmission is now evolving with a specific 55 
focus on applications and by integrating a fundamental shift 
in the customer traffic profile. Driven by the growth of 
workstations, the local area networks (LAN) interconnec- 
tion, the distributed processing between workstations and 
super computers, the new applications and the integration of 60 
various and often conflicting structures — hierarchical versus 
peer to peer, wide (WAN) versus local (LAN) area networks, 
voice versus data — the data profile has become higher in 
bandwidth, bursting, non deterministic and requires more 
connectivity. Based on the above, it is clear that there is 65 
strong requirement to support distributed computing appli- 
cations across high speed backbones that may be carrying 


2 

LAN traffic, voice, video, and traffic among channel 
attached hosts, business workstations, engineering worksta- 
tions, terminals, and small to intermediate file servers. This 
traffic reflects a heterogeneous mix of: 

end user network protocols including Ethernet, Tocken 
Ring. APPN, FDDI, OSL ISDN. ATM . . . , and 

real time (steady stream traffic such as voice and video) 
and non real time (bursty nature traffic such as interactive 
data) transmissions. 

This vision of a high speed protocol-agile backbone 
network is the driver for the emergence of fast packet 
switching networks architectures in which data, voice, and 
video information is digitally encoded, chopped into small 
packets and transmitted through a common set of nodes and 
links. Although low speed links may exist, the availability of 
fiber optic links will make cost effective to have a few links 
of high speed rather that many links of low speed. In 
addition to the high speed backbone, there exists a peripheral 
network which essentially provides access to the switching 
nodes. This peripheral network is composed of relatively 
low speed links which may not use the same protocols or 
switching techniques used in the backbone. In addition, the 
peripheral network performs the task of multiplexing the 
relatively slow end users traffic to the high speed backbone. 
Thus, backbone switching nodes are principally handling 
high speed lines. The number of high speed links entering 
each switching node is relatively small but the aggregate 
throughput very high in the Giga-bits per second range. 

Throughput 

The key requirement of these new architectures is to 
reduce the end-to-end delay in order to satisfy real time 
delivery constraints and to achieve the necessary high nodal 
throughput for the transport of voice and video. Increases in 
link speeds have not been matched by proportionate 
increases in the processing speeds of communication nodes 
and the fundamental challenge for high speed networks is to 
minimize the packet processing time within each node. As 
example, for meeting a typical 100 ms delay to deliver a 
voice packet between two end users: 

A total of 36 ms might be needed for the packetization and 
play-out functions at the end points. 

About 20 ms is the unalterable propagation delay needed, 
say, to cross the United States. 

There remains 44 ms for all the intra-node processing 
time as the packet moves through the network. In a 5 nodes 
network, each node would have about 8 ms for all process- 
ing time including any queueing time. In a 10 nodes net- 
work, each node would have about 4 ms. 

Another way of looking the same constraint is illustrated 
in FIG. 1: taking a node with an effective processing rate of 
1 MIPS (Millions of Instructions Per Second), it is possible 
to fill a 9.6 kbps line with 1000 byte packets even if a 
network node must execute 833 000 instructions per packet 
processed. For a 64 kbps line the node can afford 125 000 
instructions per packet In order to fill an OC24 link, 
however, our 1 MIPS node could only execute 7 instruction 
per packet! In the latter case even an effective rate of 10-30 
MIPS would allow only 70-200 instructions per packet. 

In order to minimize the processing time and to take full 
advantage of the high speed/low error rate technologies, 
most of the transport functions provided by the new high 
bandwidth network architectures are performed on an end- 
to-end basis. This includes the flow control and error recov- 
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cry for data, the packeiization and reassembly for voice and 
video. The protocol is simplified: 

First, there is no need for transit node to be aware of 
individual (end user to end user) transport connections. 

Secondly high performance and high quality links does 
not require any more node to node error recovery or re- 
transmission. Congestion and and flow control are managed 
at the access and end points of the network connections 
reducing both the awareness and the function of the inter- 
mediate nodes. 

Packet size 

Transmission of real time data, like voice or video pack- 
ets, which must be delivered to the receiver at a steady, 15 
uniform rate (isochronous mode) requires the use of short 
packets. In another side, pure data does not have any 
problem with transit delay. They are generated in a very 
bursty and non deterministic manner. The longer packet are, 
the fewer packets per second must be switched for a given 20 
data throughput. In order to take full advantage of the 
different data packet transmission systems, the data transfer 
across the network must be done with packets of nearly the 
same size as the user packets without processing them into 
artificial lengths. As opposed to solely data networks or 25 
solely voice or video networks, the high speed network 
architectures have to support a plurality of heterogeneous 
transmission protocols operating with variable length pack- 
els. 

30 

Connectivity 

In a high speed network, the nodes must provide a total 
connectivity. This includes attachment of the customer's 
devices, regardless of vendor or protocol, and the ability to 35 
have the end user communicate with any other device. 
Traffic types include data, voice, video, fax, graphic, image. 
The node must be able to take advantage of all common 
carrier facilities and to be adaptable to a plurality of proto- 
cols; all needed conversions must be automatic and trans- 40 
parent to the end user. For example, a high speed node must 
not have any dependencies on the existence of SNA (System 
Network Architecture) equipments on a user network. It has 
to be able to offer a similar level of service in a SNA 
environment as in a non-SNA environment made of routers, 45 
Private Branch exchanges (PBXs), Local Area Networks 
(LAN) 


50 


55 


Key Requirements 

The efficient transport of mixed traffic streams on very 
high speed lines means for each communication node of the 
network a set of requirements in term of performance and 
resource consumption which can be summarized as follows: 

a very short packet processing time, 

a very high throughput, 

an efficient queue and buffer management, 

a limited number of instructions per packet, 

a minimum impact of the control flow on the user traffic, ^ 
and 

a very large flexibility to support a wide range of con- 
nectivity options. 

The high bandwidth dictates the need- of specialized 
hardware to support very fast packet handling and control 65 
protocols, and to satisfy the real time transmission needs of 
the voice and video traffic. The processing lime being the 


main bottleneck in high speed networks, most of the com- 
munication nodes today are built around high speed switch- 
ing hardware to off-load the routing packet handling and 
routing functions from the processor. 

However, on equal performances, a software approach 
represents the most adequate solution for each node to meet 
the connectivity and flexibility requirements and to optimize 
the manufacturing and adaptation costs. The line adapters, 
are based on a common hardware design and are configured 
by means of a specific prograrnrning to execute either the 
access point or inter nodal transport functions. The adapt- 
ability of the adapters to support different access protocols 
and data streams — Frame Relay, HDLC (High level Data 
Link-Control), CBO (Continuous Bit Operations), ATM 
(Asynchronous Transfer Mode), . . . — is provided by logical 
components called Access Agents. Such logical associations 
Adapter/Access Agent are specified by software, providing 
a very large flexibility at a reduced cost Each line adapter 
is automatically configured at system initiation according to: 

the adapter function, and 

the access protocol. 

Programmable High Performance Communication 
Nodes 

The throughput of a communication adapter is defined as 
the total time required to handle a data packet from the input 
to the output. However, the packet size being application 
dependent, two measures are currently used to evaluate the 
performances of adapters: 

first, the number of packets of fixed length the adapter is 
able to handle in a second (packet throughput), 

second, the number of bits per second the adapter is able 
to transmit in case of infinite packet length (data through- 
put). 

The performances depend on the hardware and on the 
processor capabilities but the main throughput limiting 
factor is the packet processing time and this processing time 
is directly related to the number of instructions required to 
handle a packet. The operations on the packets can be 
divided in two categories: 

the background process with the operations of packet 
routing, assembling — disassembling, formatting, bandwidth 
management, priority, .... This process is designed accord- 
ing to the adapter function, but in the same application, the 
background process is identical far all packets indepen- 
dently of their size. 

the buffering process with the interrupt routines. This 
operation is generic for all adapter types. The processing 
time required for this operation is directly proportional to the 
packet length. 

Tne interrupt routines are the way real time is supported 
by the processor. They must be as short as possible to avoid 
the overrun on the input device and the underrun on the 
output device. They commands the performance of the 
adapter in term data throughput (bits per second). For 
packets of infinite length, the background process disappears 
and the throughput bottleneck is the interrupt response time 
which depends of the queuing and dequeueing operations. 

In another way, to maximize the packet throughput (num- 
ber of packets per seconds that the adapter is able to 
transmit), the number of instructions required by the back- 
ground process must be reduced to a minimum. 

Non published European patent application 93480087.1 
(IBM docket FR993027) entitled Programmable High Pcr- 
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formance Data Communication Adapter for Packet Trans- 
mission Networks (prior an under Article 54(3) EPC), which 
content is herein incorporated by simple reference, discloses 
a high performance packet buffering method and system in 
a programmable data communication adapter. The adapter 5 
includes a programmable processor, for receiving and trans- 
mitting data packets of fixed or variable length. The system 
is designed to optimize the queueing and dequeueing opera- 
lions and in particular to minimize the number of instruc- 
tions to manipulate the packets. It is characterized in that it 10 
comprises: 

means for storing data packets in buffers, 

means for identifying said data packets in the buffers 

means for queueing in a local memory the packet iden- 5 

tifiers in a single instruction, 
means for dequeueing from the local memory the packet 

identifiers in another single instruction, 
means for releasing the buffers. 

Each instruction comprises up to three operations 20 
executed in parallel by said processor: 

an arithmetical and logical (ALU) operation on the packet 
identifiers, 

a memory operation on the local memory, and ^ 
a sequence operation. 

SUMMARY OF THE INVENTION 

An efficient point-to-point and multi-points routing sys- 30 
tern and method for data communication adapters in packet 
switching nodes of a high speed network is disclosed. The 
general principles of the present invention can be summa- 
rized as follows: 

data packets are never copied during their routing through 
the adapter, only packet pointers are copied for each desti- 
nation, 

each output is processed independently on a priority 
mode, 40 

no overhead generated by the multi-points routing in the 
interrupt procedures, 

release of the resources on a non priority mode (back- 
ground procedure). 

DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows the processing times (or number of instruc- 
tions per second) required in function of the different line ^ 
throughputs supported by the present invention. 

FIG. 2 shows a typical model of high speed packet 
switching network including the access and transit nodes 
claimed in the present invention. 

FIG. 3 describes a high speed Routing Point according to 55 
the present invention. 

FIG. 4 shows a programmable high performance adapter 
according to the present invention. 

FIG. 5 represents the receive and transmit data flows in a 
Trunk Adapter. 

FIG. 6 illustrates the buffer, packet and queue structures 
according to the present invention. 

FIG. 7 illustrates the Control Point Spanning Tree. 

FIG. 8a shows a graphical representation of a typical 65 
header for packets transmitted in a network such as repre- 
sented in FIG. 2. 


60 


FIG. Sb shows a graphical representation of a Multicast 
Tree Routing Field in the header represented in FIG. 8a 

FIG. 9 represents the List Pointer structure according to 
the present invention. 

FIG. 10 shows an overview of the multicast mechanism as 
claimed in the present invention. 

FIG. 11 represents the Buffer List structure of a packet 
supporting the multicasting routing according to the present ■ 
invention. 

FIG. 12 represents the Free Buffer List structure accord- 
ing to the present invention. 

FIG. 13 represents the Processor functional structure 
according to the present invention. 

FIG. 14 shows a general view of the packet processing in 
the programmable adapter according to the present inven- 
tion. 

FIG. 15 shows the packet routing and multicasting pro- 
cess according to the present invention. 

FIG. 16 shows the interrupt procedure at packet level 
according to the present invention. 

FIG. 17 shows the interrupt procedure at buffer level 
according to the present invention. 

FIG. 18 illustrates the data flow between the Specific 
Purpose Processor (SPP), the Direct Access Memory 
(DMA) and IOl device according to the present invention. 

FIG. 19 is a flow chart illustrating the release mechanism 
of the memory resources in the background procedure as 
claimed in the present invention. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENT OF THE INVENTION 

As illustrated in FIG. 2, a typical model of communica- 
tion system is made of several user networks (212) com- 
municating through a high performance network (200) using 
private lines, carrier provided services, or public data net- 
works. Each user network can be described as a set of 
communication processors and links (211) interconnecting 
large computers used as Enterprise Servers (213), user 
groups using workstations or personnel computers attached 
on LAN (Local Area Networks 214), applications servers 
(215), PBX (Private Branch eXchange 216) or video servers 
(217). These user networks, dispersed in different establish- 
ments, need to be interconnected through wide area trans- 
port facilities and different approaches can be used for 
organizing the data transfer. Some architectures involve the 
checking for data integrity at each network node, thus 
slowing down the transmission. Others are essentially look- 
ing for a high speed data transfer and to that end the 
transmission, routing and switching techniques within the 
nodes are optimized to process the flowing packets towards 
their final destination at the highest possible rate. The 
present invention belongs essentially to the latter category 
and more particularly to the fast packet switching network 
architecture detailed in the following paragraphs. 

High Speed Packet Switching Networks 

The general view in FIG. 2 shows a fast packet switching 
transmission system comprising eight nodes (201 to 208) 
each node being interconnected by means of high speed 
communication lines called Trunks (209). The access (210) 
to the high speed network by the users is realized through 
Access Nodes (202 to 205) located at the periphery. These 
Access Nodes comprise one or more Ports, each one pro- 
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viding an access point for attaching externa] devices sup- 
porting standard interfaces to the network and performing 
the conversions required to transport the users data flow 
across the network from and to other external devices. As 
example, the Access Node 202 interfaces respectively a 5 
Private Branch exchange (PBX), an application server and 
a hub through three Pons and communicates through the 
network by means of the adjacent Transit Nodes 201, 208 
and 205. 

10 

Switching Nodes 

Each network node (201 to 208) includes a Routing Point 
where the incoming data packets are selectively routed on 
the outgoing Trunks towards the neighboring Transit Nodes. 
Such routing decisions are made according to the informa- 15 
lion contained in the header of the data packets. In addition 
to the basic packet routing function, the network nodes also 
provide ancillary services such as: 

the determination of routing paths for packets originated ^ 
in the node, 

directory services like retrieving and updating informa- 
tion about network users and resources, 

the maintaining of a consistent view of the physical 
network topology, including link utilization information, 23 
and 

the reservation of resources at access points of the net- 
work. 

Each Port is connected to a plurality of user processing 
equipments, each user equipment comprising cither a source 30 
of digital data to be transmitted to another user system, or a 
data sink for consuming digital data received from another 
user system, or, typically, both. The interpretation of the 
users protocols, the translation of the users data into packets 
formatted appropriately for their transmission on the packet 35 
network (200) and the generation of a header to route these 
packets are executed by an Access Agent running in the Port 
This network layer header (800) shown in FIG. &a is made 
of Control (801), Routing (802) and Redundancy Check 
(803) Fields. 40 

The Control Fields (801) include, among other things, an 
encoded identification of the protocol to be used in inter- 
preting the Routing Field. 

The Routing Fields (802) contain all the information 45 
necessary to route the packet through the network (200) to 
the destination End Node to which U is addressed. These 
fields can lake several formats depending on the routing 
mode specified 

The Redundancy Check Fields (803) are used to check for SO 
errors in the header itself. If an error is detected, the packet 
is discarded. 

Routing Points 

FIG. 3 shows a general block diagram of a typical Routing 35 
Point (300) such as it can be found in the network Nodes 
(201 to 208) illustrated in FIG. 2. A Routing Point comprises 
a high speed packet Switch (302) onto which packets 
arriving at the Routing Point are entered. Such packets are 
received: 60 

from other nodes overhigh speed transmission links (303) 
via Trunk Adapters (304). 

from users via application adapters called Ports (301). 

Using information in the packet header, the adapters (304, 65 
301) determine which packets axe to be routed by means of 
the Switch (302) towards a local user network (307) or 


841 

8 

towards a transmission link (303) leaving the Node. The 
adapters (301 and 304) include queuing circuits for queuing 
packets prior to or subsequent to their launch on the Switch 
(302). 

The Route Controller (305) calculates the optimum routes 
through the network (200) so as to minimize the amount of 
network resources used to complete a communication path 
and builds the header of the packets generated in the Routing 
Point. The optimization criteria includes the characteristics 
of the connection request, the capabilities and the utilization 
of the Trunks in the path, the number of intermediate nodes 

All the information necessary for the routing, about the 

nodes and transmission links connected to the nodes, are 
contained in a Network Topology Database (306). Under 
steady state conditions, every Routing Point has the same 
view of the network- The network topology information is 
updated when new links are activated or new nodes added to 
the network. Such information is exchanged by means of 
control messages with all other Route Controllers to provide 
the necessary up-to-date information needed for route cal- 
culation (such database updates are carried on packets very 
similar to the data packets between end users of the net- 
work). The fact that the network topology is kept current in 
every node through continuous updates allows dynamic 
network reconfigurations without disrupting end users logi- 
cal sessions 

The incoming transmission links to the packet Routing 
Point may comprise links from external devices in the local 
user networks (210) or links (Trunks) from adjacent network 
nodes (209). In any case, the Routing Point operates in the 
same manner to receive each data packet and forward it on 
to another Routing Point as dictated by the information in 
the packet header. The fast packet switching network oper- 
ates to enable a communication between any two end user 
applications without dedicating any transmission or node 
facilities to that communication path except for the duration 
of a single packet. In this way, the utilization of the com- 
munication facilities of the packet network is optimized to 
carry significantly more traffic than would be possible with 
dedicated transmission links for each communication path. 

Control Point Spanning Tree 

Associated with networks are several network control 
functions: network Spanning Tree maintenance, Topology 
Database, Directory, Path Selection, Bandwidth Manage- 
ment and Reservation, and Congestion Control. Preferably, 
every node has a set of the foregoing network control 
functions called its Control Point (CP) that the node uses to 
facilitate the establishment of connections between user 
applications. 

The main purpose of the Control Point Spanning Tree is 
to ensure a communication and distribution mechanism for 
all network control functions in the nodes of a high speed 
network. It (logically) joins together the Control Points 

(305) if the nodes are in a (physically) connected portion of 
the network. As illustrated in FIG. 7 a tree is a pattern of 
connections with no loop, the term "spanning" means that 
the tree spans (connects) all of the nodes. Once formed, the 
Control Point Spanning Tree is the principal system used to 
disseminate control information such as Topology Database 

(306) updates. This mechanism is fundamental to minimize 
delays due to intermediate node processing. 

First, an intermediate node will get each control message 
exactly once on the tree, and 

second, the message can be forwarded along outgoing 
links of the tree before the intermediate node has even 
looked at the packet contents. 
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Multi-Points Transmission 
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A distributed algorithm creates and maintains the Control 
Point Spanning Tree in presence of node and link failures 
and helps to minimize the impact of the increased control 
flows that result when the network grows. 

Routing Modes 

The routing within the network presents two aspects: 

1. Determining what the route for a given connection shall 

be. 

2. Actually switching the packet within a switching node. 
There are many methods of determining a route through 

a network. For veiy high throughput, once the route selected, 
the critical item is that the switching elements must be able 15 
to route an incoming packet in a very short portion of time. 
Driven by the requirements to keep transit node processing 
at a minimum, the transport services are designed to operate 
on an cnd-to-end basis so there is no hop-by-hop error 
recovery or retransmission envisioned for high speed, high M 
performance (low error) links. There is also no need for 
transit nodes to be aware of individual transport connec- 
tions. 

Data packets are routed and queued in the transit nodes 
according to the routing information contained in the header. 25 
Several routing modes can be used in high speed networks 
(refer to an Introductory Survey (pages 1 1 6 to 1 29)— GG24- 
3816-01 ITSC Raleigh June 1993). However, in most of the 
case, packets using different modes can share the same data 
transmission facilities. 30 

To classify the different transport type, we can consider on 
one side the point-to-point transmissions on the other side 
the multi-pointss transmissions. As illustrated by the fol- 
lowing examples, each routing mode has its particular 
indented use and includes advantages and disadvantages that 35 
complement the other modes.: 

PoinMo-Point Transmission 

Source Routing 40 

The Source Routing is a particular implementation of the 
distributed routing for connectionless networks. The The 
source node (or access node) is responsible for calculating 
the route the packet must take through the network. 

Each packet includes in its routing field a list of the labels 45 
of all links through which the packet will pass as it moves 
across the network. The Source Routing requires no con- 
nection setup activity in intermediate nodes and supports 
true datagram services. 

Label Swapping 50 

This routing mode is used in connection oriented net- 
works. Each packet sent on the link has a header which 
includes an arbitrary number identifying which logical con- 
nection that this packet belongs to. Label Swapping requires 
that connection tables be set up and maintained dynamically 55 
in each node. Due to the low packet overhead, this technique 
is particularly adapted to the transmission of very short 
packets (for example real-time voice connections). 
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Multicast allows one entity to communicate with multiple 
entities. An efficient multicast mechanism provides packet 
delivery to a set of users without having to broadcast to all 
users in the network or having to "unicast" separate copies 65 
of packets to each user of a group. 
Multicast Tree Routing 


Multicast Tree Routing is supported by the ability of each 
node to recognize that a particular label represents a pre- 
defined tree and to forward the received packet along all the 
outbound links associated with that tree. This is the basic 
routing mechanism that underlies both the fundamental 
Control Point Spanning Tree and any individual trees that 
can be established, pruned, and removed dynamically in a 
network. The Control Point Spanning Tree greatly reduces 
the negative effect on performance that network control 
flows could have in a high speed network. Tree routing on 
other Multicast Trees can be fundamental to supporting, for 
example, multi-attached LANs, or specific applications like 
video conference. 

Remote Access to a Multicast Tree 

Remote Access to a Broadcast Tree is a straightforward 
combination of the two routing modes: Source Routing and 
Multicast Tree Routing. The routing field includes several 
Source Routing labels followed by a tree address label. This 
allows a node that is not a member of a tree to route packets 
to nodes that are tree members. This technique is described 
with more details in European patent application 
93480059.0 filed May 19th 1993 entited Method and Appa- 
ratus for Routing Packets in Packet Transmission Networks. 


Multicast Mechanism 

The Multicast Tree Routing provides a broadcast capa- 
bility over a preselected tree that can connect multiple 
nodes. Any member of a multicast group can send a packet 
which will be forwarded to all other member of the group. 
A Multicast Tree is simply identified, in the Routing Point, 
by a reference in the Topology Database. Different trees can 
coexist in the same network. One internal use of a Multicast 
Tree is to join all the Control Points of a network (Control 
Point Spanning Tree) for maintaining in each Topology 
Database a true image of the network configuration. This 
broadcast mechanism can also be used to address subset of 
network users Qikt a closer user group). 

As illustrated in FIG. 8i>, a Multicast Tree is identified by 
a tree address (804) which is pan of the routing field (802) 
of packets to be sent on the tree. Unlike the Source Routing 
labels which are uniquely preassigned to individual links 
within each node, the tree address is assigned to all links that 
are to be part of a tree at the time that the tree is set up. The 
tree address must be unique to one and only one tree across 
all nodes over which the tree passes. When a packet 
addressed to a Multicast Tree arrives at a node over a link, 
any links in that node that are configured for that tree address 
will forward this packet. In this manner, copies of the packet 
flow across the network, along each link configured as a 
branch of the tree, with one and only one copy arriving at 
each node that is part of the tree. A Hop Countdown Field 
(805) is included in the Multicast Tree Routing Field (802) 
which is decremented at each retransmission of the packet. 
When the hop countdown is equal to zero, no further 
retransmission is permitted and an error condition is 
assumed. The routing field is tenninated by and End Of Field 
(EOF) flag (806). 

Any node can be simultaneously be a member of a 
plurality of different trees and hence the tree addresses 
assigned to overlapping multicast trees must be unique. 
Nodes can be added or deleted from a Multicast Tree simply 
by adding or removing the tree address from the links 
leading to the node to be added or deleted. A more detailed 
description is disclosed in European patent application 
93480060.8 filed May 19th 1993 entitled Multicast Com- 
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munication Tree Creation and Control Method and Appa- 
ratus. 

Ports and Think Adapters 

5 

Adapters Function 

Ports are located at the boundary of the high speed 
network. They allow terminal equipments to exchange infor- 
mation through the high speed network without the need for 10 
knowing the specific high speed protocol used. The main 
function of the Ports are: 

receiving foreign protocol data units from an external 
resource and forwarding them as high speed packets over the 
network to a target Port, and is 

converting high speed packets back to foreign protocol 
data units and sending them to the target resource, 

controlling the bandwidth. 
Note: the source and target Ports may be located in the same 
node. 20 

Trunks are the links between the high speed network 
nodes. They carry high speed packets. Each Trunk manage 
its link bandwidth and link status. A critical task of the 
Trunks is the management of traffic priorities and allocation 
of internal buffers to reduce delay and congestion. 25 

In addition, there is a special type of adapter called Route 
Controller Adapter (305) which: 

communicates with the other adapters (301, 304) through 
the Switch (302), 

implements the centralized functions of the Rome Con- 30 
troller such as the topology, the path selection . . . , 

establishes, maintains and cancels end-to-end high speed 
connections. 


Adapters Architecture 


35 


Several techniques for designing said Ports, Trunk and 
Route Controller Adapters, to obtain more or less flexible 
and efficient transmission systems. Most of the adapters 
today are built around a specialized hardware depending on 40 
the function and protocol of the connected links. 

The present invention, to satisfy the previously enumer- 
ated connectivity and flexibility requirements, provides a 
software solution based on a common hardware structure. 
Port and Trunk Adapters present the same architecture and 4S 
their functional differentiation is realized through a specific 
programming. However, even using the most efficient gen- 
eral purpose microprocessor today available on the market, 
the experience shows that it is very difficult to reach the 
desired level of performance in term of number of switched 50 
packet per second. This is the reason why the control of each 
adapter has been shared between two processors: a Specific 
Purpose Processors (SPP. 406, 412), optimized for the 
packet switching and a General Purpose Processor (GPP, 
409), the first dealing with the packet to be switched, the 55 
critical processing in term of performance, and the second 
with the adapter management. 

As shown in FIG. 4, each adapter (400) comprises the 
following logic components: ^ 

1. A General Purpose Processor (GPP, 409) whose pro- 
gramming depends of the selected Port or Trunk Adapter 
function. The GPP implements the adapter control opera- 
tions. 

2. A Receive Adapter (401) for implementing three tunc- fi5 
lions: 

the checking of the high speed packets header. 


the traffic discrimination according to the routing mode 
specified in the header of every incoming packet, 

the routing of the incoming packets through the Switch 
(403) with the appropriate header. 

The Receive Adapter includes: 

a. a Line Receiver (407) for handling the data movements 
between a Line Interface (415) and a Receive Buffer 
Memory (RBM, 405). 

b. a Receive Buffer Memory (RBM, 405) to temporarily 
store users data packets. 

c. a Receive Specific Purpose Processor (RSPP, 406) 
based on a specialized microprocessor comprising a Local 
Memory (LM, 408). The RSPP handles the received steady 
slate packet flow and forwards the control packets to the 
General Purpose Processor (409). 

d. a Local Memory (LM, 408) used by the RSPP (406) as 
worit space. 

e. a Switch Transmitter Adapter (404) for 

handling the data flow transferred from the buffer 
Memory (RBM, 405) under the control of the Receive 
Specific Purpose Processor (406) 

segmenting this flow in fixed length cells and, 

generating an appropriate switch routing header 

3. a Transmit Adapter (402) for implementing the follow- 
ing functions: 

ihe reception of the data flow from the Switch (403), 

the checking of the cells header. 

the reassembly in packets (Port), 

the Trunk functions (Trunk adapter), 

the routing. 

The Transmit Adapter includes: 

a. a Switch Receiver (410) for handling the flow coming 
from the Switch (403) and transferring it to the Buffer 
Memory for reassembly. 

b. a Transmit Specific Purpose Processor (XSPP, 412) 
similar to the Receive Specific Purpose Processor (406). The 
XSPP handles the steady state data and forwards the control 
flow to the General Purpose Processor (GPP, 409). 

c. a Line Transmitter Adapter (413) for handling the data 
movements between the Buffer Memory (411) and the Line 
Interface (415). 

The adapters are connected on one side on the packet 
Switch and on the other side on the Line Interfaces: 

The Line Interfaces (415) are used to adapt the Port and 
Trunk Adapter physical interfaces to the appropriate media. 

The packet Switch (302, 403) allows the Route Controller 
(305) and the different Ports (301), Trunk Adapters (304) to 
communicate. 

Data Flow Control 

The receive and transmit data flows in the Trunk Adapters 
are represented in FIG. 5. In a proprietary high speed 
network with packets of variable lengths, the receive process 
involves the steps of: 

1. Line Receiver, Buffer Memory, Specific Purpose Pro- 
cessor (501) system 

a. receiving the packets from the line, 

b. checking the packets header and in case of error 
discarding the packets, 

c. processing the information contained in the packets 
header according the routing mode, 

d. routing the control messages towards the General 
Purpose Processor (GPP, 502) 
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e. encapsulating the packets with a specific switch header 
in function of the destination adapter, 

f. forwarding the packets and the GPP (502) control 
messages to the Switch Transmitter Adapter (504), 

2. Switch Transmitter Adapter (504) 5 

a. segmenting the packets in cells of fixed length adapted 
to the Switch (503), 

b. generating an error check field to ensure the integrity, 

of the switch header during the transmission of the cells over 10 
the Switch (503). 
The transmit process, as for it, comprises the steps of: 

1. Switch Receiver Adapter (505) 

a. receiving the cells from the Switch (503), ^ 

b. checking the switch header and, in case of error, 
discarding the cell, 

2. Line Receiver, Buffer Memory, Specific Purpose Pro- 
cessor (506) system 

a. reassembling the data packets, 20 

b. forwarding the control packets to the General Purpose 
Processor (502), 

c. encapsulating the packets with a routing header, 

d. receiving control packets from the GPP (502), M 

e. queueing data and control packets in the appropriate 
queues, 

f. handling the outgoing packets with priority given to real 
time data (and then to non real time data). 

It is possible to design the adapters to work either in a a 30 
proprietary environment with packets of variable length, or 
in a standard mode such as ATM (Asynchronous Transmis- 
sion Mode) with short cells of fixed length, or in both where 
appropriate. In this last case, for performance purpose, cells 
routed on the switch are identical or similar to these defined 33 
in the ATM protocol with as result: 

the elimination of the packets segmentation and reassem- 
bly steps in the Switch Transmitter (508) and Receiver 
Adapters (509), 

a simplification of the switch header processing in the 
Specific Purpose Processor (507, 510). 

Adapter Functional Structure ^ 

The present invention deals with the relationships 
between the Line Receiver/Transmitter, the Buffer Memory, 
the Specific Purpose Processor, and the Switch Adapter and 
in particular with the handling of the data flow in a way to 
optimize the throughput and the processing time inside the 50 
adapters. More specifically, the invention relates to a very 
high performance system for queuing, dequeueing and dis- 
tributing data packets on external links. 

The communication adapters are based on the following 
principles: 55 

the Specific Purpose Processor is designed to minimize 
the number of operations necessary to manage the steady 
state data flow. 

data packets and control data are managed separately in ^ 
two distinct memories respectively the Buffer Memory and 
the Local Memory. 

the data packets buffering, the queueing, dequeuing, the 
routing and multicasting mechanisms are identical for all 
Ports— Trunk Receive and Transmit adapters. 65 

According to these considerations, the following conven- 
tions will be used to describe the invention: 
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The device which reads in the Buffer Memory and the 
device which writes into the Buffer Memory are designated 
respectively by IOl and 102. That means: 

in the receive side of the adapter, I01=Switch Transmitter 
and I02=Line Receiver 

in the transmit side of the adapter, I01=Line Transmitter 
and I02=Switch Receiver. 
In the same way: 

an input data stream goes from the switch or external line 
(102) to the Buffer Memory. 

an output data stream goes from the Buffer Memory to the 
switch or external line (IOl). 
Furthermore: 

The meaning of "packet" is application dependent. It may 
be applied, for example, to an SDLC-frame from a Port, to 
a proprietary packet format from a Trunk, or to a cell 
received from an ATM Trunk. The term "packet" that will be- 
used in the following paragraphs will not refer to a precise 
data unit format. 

Data Structures 

Buffers, Packets, Queues Structures 

Data packets are stored in the Buffer Memory (BM) while 
control data are managed directly by the Specific Purpose 
Processor (SPP) in the Local Memory (LM). The basic unit 
of memory that can be allocated to an input (Line/Switch 
Receiver (102)) or output device (Line/Switch Transmitter 
(IOl)) is a buffer of fixed length. As illustrated in FIG. 6, 
each of these buffers is represented in the Local Memory by 
a pointer called Buffer Pointer (B_PTR). A pointer is a 
generic term to identify a logical data structure (buffer, 
packet, queue . . .) stored in the Buffer Memory. The storage 
of a packet requires one or several buffers. These buffers are 
stacked together U3ing a list of pointers (B_L1ST) which is 
itself represented by a Packet Pointer (P_PTR). A list of 
Packet Pointers (P_L1ST), identified by a Queue Pointer 
(Q_PTR), designates a queue of several packets. 

List Prefix 

Each list, representing a specific packet or a queue struc- 
ture, is preceded by a Prefix used for storing any type of 
information related to the data the structure contains. In 
Buffer Lists, the Prefix contains information related to the 
routing of the packet; 

the packet header 

the date of the packet reception 

the packet length 

All the processor operations on the packet header are 
realized in the the Local Memory (LM) without having to 
access to the data stored in the Buffer Memory (BM). 
Furthermore, when the processor (SPP) is working on the 
Local Memory, the DMA operations on the Buffer Memory 
are not disrupted. The result is a more efficient routing 
process and memory management As illustrated in FIG. 11, 
the support of the multicasting mechanism is realized by 
means of specific fields in the Buffer List Prefix of a each 
packet: 

a Multicast Counter (MCC) 

The counter is initialized according to the number of 
outputs. The intitial counter value is one or superior to one 
depending on if the packet is intended to be transmitted to 
a single (MCC=1: Point-to-point Routing Mode) or to 
multiple destinations (MCC>1: Multi-points Routing 
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Mode). The counter value is decremented each time the 
packet to multicast is transmitted to an output. When the 
counter value is equal to zero, the routing process is termi- 
nated and the resources attached to the packet can be 
released 

a copy of the Packet Pointer (P_PTR) 

The original Packet Pointer, with the initial HEAD (iden- 
tification of the first Buffer Pointer in the list) and TATJL 
(identification of the next Buffer Pointer in the list) values, 
is saved in the Buffer List Prefix to allow, at the end of the 
transmission, the complete release of all Buffers attached to 
the packet 

Packet Segmentation 

To facilitate the memory management, the lists used for 
packets and queues are of fixed length. Packets which are 
bigger than the buffer list can contain, are segmented. This 
method allows the lists (B_LIST) not to be sized at the 
maximum packet length. 

Buffer Pointers 

Buffers need not to be full and the data may start and end 
at any place. For example, it is possible to reserve the first 
bytes of a buffer to include a header a posteriori. A Status 
Field (SF) is used in the last Buffer Pointer of a list to 
designate an End Of Packet (EOP). The general format of a 
Buffer Pointers is described in European patent application 
93480087.1 (IBM docket FR993027) entitled Program- 
mable High Performance Data Communication Adapter for 
Packet Transmission Networks 

List Pointers 

Referring to FIG. 9, the List Pointer format consists of 35 
three fields: 

the List IDentifier (LTD): identification of the list, 
the HEAD: identification of the first pointer of the list, 
the TAIL: identification of the next pointer to attach to the 
list. 

The queuing process comprises the steps of: 
testing if list is full (if INCREMENTED(TAIL)=HEAD). 
storing the pointer at the address identified by the TAIL 

field in the pointer list identified by the LID field, 
incrementing the TAIL field of the List Pointer, 
The dequeueing process comprises the steps of: 
testing if the list is not empty (if HEAD not equal to 

TAIL). 

reading the pointer at the address identified by the HEAD 
field in the pointer list identified by the LID field, 

incrementing the HEAD field of the List Pointer when list 
is not empty. 

This queueing and dequeueing method is described with 
more details in European patent application 93480087.1 
(IBM docket FR993027). 
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HEAD: identification of the first free buffer of the list. 
Each time a new buffer is filled, the HEAD field is incre- 
mented. 

TAIL: identification of the next free buffer of the list Each 
time a new buffer is released, the TAIL field is incremented. 

HEAD=TAIL: the Buffer Memory is full. 

Incremented TAIL=HEAD: the Buffer Memory is empty. 

The Free Buffer List (FBL) is created at initiation time 
and is permanent contrary to other lists which arc created 
dynamically (Buffer, Packet or Queue Lists). 
Note: in general, when a lack of resources is detected (Free 
Buffer List empty), then the packet which cannot be stored 
in the Buffer Memory is discarded 

Free Packet List (FPL) 

Buffer Lists are of fixed length (fixed number of ele- 
ments). The management of these lists in the Local Memory 
(LM) is realized by means of a specific list called Free 
Packet List (FPL). The FPL is created at initiation time and 
is permanent like the Free Buffer List It comprises all the 
Buffer List structures that will be dynamically created or 
released during the packet receive and transmission process. 

It is important to note that, in the present invention, 
pointers (Buffer, Packet or Queue pointers) are simply 
stacked in predefined lists (FIG. 6). The pointers are not 
chained that means that pointers do not contain the address 
of the next pointer in the list and they are not aware of the 
location of this next pointer. This list structure is designed to 
optimize in term of performance the buffer manipulation in 
very high speed adapters but not the memory resources. 

Specific Purpose Processor Structure 

The Specific Purpose Processor functional structure is 
illustrated in FIG. 13. 

Processor Parallel Processing 

The Specific Purpose Processor is designed to execute up 
to three operations in parallel: 

1. ALU (Arithmetical and Logical Unit) operations on 
registers 

2. Memory operations 

3. Sequence operations 

The parallelism requires to distinguish instructions from 
operations: 

Instruction: it is the content of the code word. In term of 
assembler language, the instruction corresponds to one line 
of code. All instructions are executed in one processor cycle 

Operation: an instruction may be composed of one or 
more operations which are executed simultaneously. 
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Free Buffer List (FBL) 


60 


The management of the Buffer Memory is realized by 
means of a specific list called Free Buffer List (FBL). The 
FBL comprises the totality of the Buffer Pointers and its role 
is to provide a status of the memory occupation (FIG. 12) 
using the HEAD and TAIL fields of the Free Buffer List « 
Pointer (P_FBL): 

T: total number of buffers in the Buffer Memory. 


Memory Space 

The SPP memory space is divided in three blocks: 
the Instruction Store (130), 

The Local Memory (LM, 131), which is the code working 
space, 

The Buffer Memory (BM, 132) which is the repository for 
data packets when they pass through the adapter. 

They all operate in parallel, the Instruction Store (130) 
under control of the Sequencer (133), the Local Memory 
under control of the processor code and the Buffer Memory 
(132) under the control of the Direct Access Memory (DMA, 
134)). 
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Registers 

The registers are divided in two categories: 

1. the General Purpose Registers (GPR) 

These registers are located in the Register Hie (RF, 135) 
and are available as instruction operands. 

2. the Control Registers (CR) 

The CR's are hardware regis ten which are used in 
specific functions and are available also as instruction oper- 
ands. However, there are less degree of freedom in their use, 
as compared with the GPR's. In particular, two of these 
control registers (136) are located in the Direct Access 
Memory (DMA, 134). 

CR1=D_PTR1 (DMA Pointer 101) 

CR2=D_PTR2 (DMA Pointer 102) 

DMA Pointers 1 and 2 are associated to input/output 101 
(137) and 102 (138) and they both contain the current Buffer 
Pointer (B_PTR). 

Memory Address Generator (MAG, 139) 

In all load or store operations, on the Local or on the 
Buffer Memory, the physical address is reconstituted from 
the different fields of Buffer or List Pointer, used as operand. 
For performance reason, this operation is realized by a 
specialized hardware component called Memory Address 
Generator (MAG.139). 

Direct Memory Access Controller (DMA, 134)) 

The use of a Direct Memory Access Controller (DMA, 
134) jointly with a processor is well-known in the state of 
the art. Its role is to quickly move the data packets between 
the 10 devices (137, 138) and the Buffer Memory (132) 
without the processor (SPP) intervention. The DMA module 
consists of two independent programmable channels. The 10 
devices present their service requests (SRI, SR2) to the 
DMA which controls the access to the Buffer Memory (132). 
The processor intervention is needed only at buffer and 
packet boundaries. The data streams between the two 10 
devices and the Buffer Memory is processed in parallel with 
the code. Up two 10 operations can be multiplexed on the 
BMIO bus; one with 101 and the other with 102. For that the 
DMA manages the two DMA Pointers (D_PTR1 and 
D_PTR2) which are nothing else than Buffer Pointers. 

Input Output Subsystems 

The Specific Purpose Processor (SPP) is considered as the 
"master" and it establishes the connections. 

The 10 devices and the Buffer Memory are controlled 
either directly by the processor code, or via the DMA. 

The code intervention can be forced by the 10 devices via 
the Interrupt mechanism in case of buffer or packet bound- 
ary. 

Data Reception and Transmission 

Various processing can be made on Buffer and List 
Pointers: 
Incrementing a Buffer Pointer, 
Closing a buffer, 
Accessing a List Prefix, 
Attaching an element to a list, 
Detaching an element from a list 
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Operations on Pointers 

Some operations on pointers are performed by the pro- 
cessor code, others by the Direct Memory Access (DMA). 

The writing and reading of the Buffer Pointers is exclu- 
sively the fact of the DMA (134). 
The writing in the Buffer Memory: At the reception of a 
service request (SR2) from an 102, the DMA has access 
to the Buffer Memory (132) by means of the address 

10 contained in the Pointer 2 (D_PTR2, 136). The DMA 
Pointer 2 is provided by the processor (SPP) and is 
nothing else than a Buffer Pointer (B_PTR). The DMA 
orders simultaneously the 102 (138) to present a data 
element on the BMIO bus and the Buffer Memory (132) 

15 to write this data element in the buffer identified by the 
DMA Pointer. Data are filled into the buffer starting an 
address chosen by the code and ending at the bottom of 
the buffer except the last buffer (of a packet) where data 
may end at any place. When the buffer is full, the DMA 

20 demands from the processor a new Buffer Pointer 
through an interrupt mechanism (102_E0B routine), A 
similar procedure is used when the 102 detects the end 
of a packet (I02_EOP routine)). 
The Reading in the Buffer Memory: At the reception of a 

23 service request (SRI) from an 101, the DMA addresses 
the Buffer Memory (132) by means of the DMA Pointer 
1 (DJTRI, 136). The DMA Pointer 1 is provided by 
the processor (SPP). The DMA orders simultaneously 
the Buffer Memory (132) to present on the BMIO bus 

30 a data element in the buffer identified by the DMA 
Pointer and the 101 device (138) to read this data 
element When the buffer is empty, the DMA demands 
a new Buffer Pointer from the processor through an 
interrupt mechanism (I01__E0B routine). A similar 

35 procedure is used when the DMA detects a end of 
packet (I01_E0P routine). After the data transfer, the 
Buffer Pointer is released in the Free Buffer List, Packet 
List and Queue List arc updated accordingly. 
Packet and Queue Pointers are managed by the processor 

"0 code: 

the Code intervention is forced by the 10 devices and 
DMA via the Interrupt mechanism in case of buffer or 
packet boundary. 
45 the buffer and packet queueing and dequeueing mecha- 
nisms are executed under the control of the processor in 
the Local Memory. 


50 . Interrupts 

The Interrupt mechanism is the way real time is supported 
by the Specific Purpose Processor (SPP). An Interrupt is a 
break, due to particular events, in the normal sequence of the 
processor code. The events which can cause this Interrupt 
35 are the service requests from the IO devices associated with 
specific conditions such as end of buffer, end of packet . . . 
. At each specific Interrupt corresponds a specific routine 
which cannot be interrupted by another routine. It is very 
important to have interrupt routines as short as possible to 
60 avoid overrunAmdenun problems. 

1. The following Interrupts are used by the Line and 
Switch Transmitters (IOl). 
I01_E0B: 

65 Condition: when serving an output 101, the DMA has 
emptied a buffer (which is not the last buffer of the packet/ 
segment — the Buffer Pointer is not flagged EOS or EOP) the 
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DMA Pointer (D_PTR1) raises a I01_EOB interrupt which 
triggers a I01_EOB routine- 
Routine: this routine 

releases the pointer of the just emptied buffer in the Free 
Buffer List- A new pointer is dequeued from the Output 
Buffer Ust (OB_LIST) and passed to the DMA Pointer 
(D_PTR1). 

I01_EOP: 

Condition: when serving an output IOl, the DMA has 
emptied a packet (the last buffer of a packet which pointer 
contains the EOP flag ON), the DMA Pointer (D_PTR1) 
raises a I01_EOP interrupt which triggers a I01_EOP 
routine. 

Routine: the routine: 

releases the current and last Buffer Pointer of the Output 
Buffer List (OB_LIST) in the Free Buffer List (FBL). 

releases the current Packet Pointer in the Free Packet List 
(FPL) 

detaches the current Output Packet Pointer (OP_PTR) 20 

from the Output Packet List (OP_UST). 
dequeues the next packet and its pointer from the Output 

Packet List 

2. These Interrupts are used by the Line and the Switch 
Receiver (102). 
I02 13 EOB: 

Condition: when serving an input 102, the DMA has 
detected a Buffer Full condition (That buffer is not the last 
buffer of a packet), the DMA Pointer (D_PTR2) raises a 
I02_EOB interrupt which triggers a I02_EOB routine. 30 

Routine this routine: 

stores the pointer of the just filled up buffer in a pre- 
allocated Input Buffer List (IB_UST) where all buffers 
of the same packet are stacked. 

updates the Input Buffer List Prefix area. 

provides the DMA Pointer (D_PTR2) with a new Buffer 
Pointer for continuing the reception of the data of the 
same packet. Free Buffer Pointers are managed by the 
Free Buffer List (FBL). - 40 

when for the same packet, the Buffer List is full, the 
segmentation takes place. 

I02_EOP: 

Condition: when serving an input 102, the DMA has 
completed the reception of a packet (the last buffer of the 45 
packet which pointer is flagged EOP by the hardware), the 
DMA Pointer raises a I02_EOP interrupt which triggers a 
I02_EOP routine. The code intervention is required to 
provide a new Buffer Pointer for the reception of a new 
packet. 50 

Routine: this routine: 

stores the pointer of the last filled buffer in a pre- allocated 
Input Buffer List (1B_LIST) where all buffers of the 
same packet are automatically stacked. The last Buffer 
Pointer of a packet is nagged EOP by the DMA Pointer 55 
(D_PTR2). 

updates the Input Buffer List Prefix area. 

the Packet Pointer of the Input Buffer List is queued in the 
Input Packet List (1P_LIST). m 

provides the DMA Pointer (D_PTR2) with a new Buffer 
Pointer (B_PTR) for the reception of the next packet. 


35 


Point-to-Point and Multi -Point Routing 

Multicasting means a packet should be sent to at least two 
destinations: 


65 


to different Transmit Adapters (via the Switch) in a 
Receive Adapter and when needed to the General Purpose 
Processor (GPP). 

to output Trunks or Ports in a Transmit Adapter and when 
needed to the General Purpose Processor (GPP). 

The general principles of the point-to-point and multi- 
points routing according to the present invention can be 
summarized as follows: 

Data packets are never copied, only Packet Pointers are 
copied for each destination: 

Space in Buffer Memory is saved. 

The number of instructions required for the background 
(delayed) process is significantly reduced improving 
the packet throughput (number of packets per seconds 
that the adapter is able to transmit). 

The routing is independent of the packets length. 

No overhead generated by the multicasting mechanism in 
the interrupt (real time) procedures. 

The underrun/overrun problems on the output device IOl 
are reduced. 

The efficiency of the adapter in term data throughput (bits 

per second) is significantly improved. 
Each output is processed independently by means of 
interrupt (real time) routines: 
Lines are managed in real time. 

Lines of different speed or protocol can be supported in 
parallel. 

The release of the resources is entirely realized on a no 
priority mode in the background (delayed) process. 

Output Queues 

Each destination has its own Output Queue where the 
packets waiting for transmission are stacked. The number of 
queues is limited to the number of destinations: 

In the Receive Adapter (401), there is one Output Queue 
per adapter on the Switch (403) plus one specific Output 
Queue dedicated to the General Purpose Processor (GPP, 
409). 

Likewise in the Transmit Adapter (402), there is one 
Output Queue per Trunk or line in output of the Line 
Interface (416) plus one specific Output Queue dedicated to 
the General Purpose Processor (GPP, 409). 

The Output Queues are located in the Local Memory 
(131). They are managed by the Specific Purpose Processor 
(SPP) by means of the background and interrupt (EOB, 
EOP) procedures detailed in flow charts of FIGS. 15, 16 and 
17. 


Output Queue Table 

As shown in FIG. 10, during the transmission of the data 
packets, the state of each Output Queue is stored in a specific 
table called Output Queue Table (100). Each Output Queue 
is represented by one record comprising: 

the Queue Pointer (104)(Q_PTR) identifying the queue. 

the Current Packet Pointer (105) (cur P_PTR) identify- 
ing, when the Output Queue is ACTIVE, the packet to 
transmit. 

the Current Buffer Pointer (106) (cur B_PTR) identify- 
ing, when the Output Queue is ACTIVE, the buffer to 
transmit. 

the stale (107) (ST) of the Output Queue: 
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ACTIVE state: a START TRANSMISSION has been sent 
to the 101. The packet identified by the Current Packet 
Pointer (cur P_PTR) is transmitted from the Buffer 
Memory to the selected output under the control of the 
DMA. 

NON ACTIVE state: the transmission of the previous 
packet is terminated and the Output Queue is empty. 

Packet Processing 

FIG. 14 gives a general view of the packet processing 
mechanism in a programmable high speed adapter. The 
receive, routing and transmission process involves the steps 
of: 

(140) Reassembling each packet received in an Input 
Buffer List and initializing the Buffer List Prefix with the 
routing information. Packets in error are discarded. 

(141) Creating for each packet a specific Buffer List 
identified by a Packet Pointer (I PI . . . Ipm). 

(142) Processing the packets according to their routing 
mode with a local access to a Routing Table (148). 

(143) Queueing the Packet Pointers in the Output Queues 
(OQ1 . . . OQN) corresponding to the destination of the 
packets (145). Packets intended for the GPP (146) are 
stacked in a specific Output Queue (144) 

(149) Triggering an interrupt routine each time a new 
buffer is requested. Interrupts are triggered asynchronously 
by the DMA on the occurrence of specific events: End Of 
Buffer (EOB), End Of Packet (EOP). 

(1410) Transmitting the data from the Buffer Memory 
(BM.147) to the output device (101) under the control of the 
DMA. 

(1411) Scanning the outputs (145). 

Background Procedure 

FIG. 15 is a flow chart illustrating the the packet routing 
and multicasting procedure represented in FIG. 14 (142): 

(150) The processor waits for the next reassembled 
packet 

(151) As soon a packet is ready to be processed, its Packet 
Pointer (P_PTR) is retrieved. The Buffer List Prefix iden- 
tified by said Packet Pointer (P_PTR) is updated. FIG. 10 

' details the different elements of the routing mechanism: 
(102) The determination of the different destinations is 
given through a Routing Table (156) accessed by the 
Routing Field (802) of the packet header. In case of 
multi-points routing, Multicast Trees are predefined in 
each nodes at the connection set-up. Routing Tables 
(156) are maintained dynamically. That means that 
when a new connection is established or an old one is 
terminated the tables are updated (the database of 
network topology can of course be maintained quite 55 
separately). The Multicast Counter (MCC) in the Buffer 
List Prefix is set to its initial value according to the 
number of destinations given in the Routing Table 
(156). This value is equal to one for a point-to-point 
routing and superior to one for a multi-points routing go 
mode. 

(101) The Packet Pointer (P_PTR) (with the original 
HEAD and TAIL values) is saved in the Buffer List 
Prefix for a further release of the Buffers in the Buffer 
Memory. 

(152) The Packet Pointer (P_PTR) corresponding to the 
packet to route or multicast is queued (the TAIL value of the 
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Queue Pointer is incremented) in the first selected Output 
Queue (103). 

(153) If the Output Queue is in an ACTIVE state, the 
process start again with with the next selected Output 
Queue (if any). 

(154) If the Output Queue is in an NON ACTIVE state 
(ST), the first Packet Pointer of the Output Queue is 
dequeued (the HEAD value of the Queue Pointer is 
incremented) and saved as Current Packet Pointer (cur 
P_PTR) in the Output Queue Table. Likewise, the first 
Buffer Pointer (B_PTR) of the corresponding Buffer 
List is dequeued (the HEAD value of the Packet Pointer 
is incremented) and saved as Current Buffer Pointer 
(cur B_PTR) in this same Output Queue Table. Then, 
the Specific Purpose Processor (SPP) sends to the 101 
a request to start the transmission of the Current Packet 
(START TRANSMISSION). The Output Queue is set 
in an ACTIVE state (ST) in the Output Queue Table, 
The process goes on with the next selected Output 
Queue (if any). 

(155) When all the selected outputs have been succes- 
sively processed the packet routing and multicasting proce- 
dure releases all resources attached to the packets and 
buffers already transmitted (157). It is important to note that 
this release takes place in the background procedure without 
overloading the interrupt procedures. 

(150) Once the release terminated the procedure returns to 
its initial waiting state. 

Interrupt Procedures 

1. Packet Level (I01_EOP) 

FIG. 16 is a Row chart illustrating the interrupt procedure 
at the packet level as represented in FIG. 14 (149). On the 
occurrence of an End Of Packet interrupt (EOP): 

(160) The Current Packet Pointer (cur P_PTR) corre- 
sponding to the last Output Queue processed (OQn) is 
extracted from the Output Queue Tabic. 

(161) The Current Packet Pointer is queued in a Release 
Queue for a further release of the Packet Pointer in the Free 
Packet List and of the Buffer Pointers in the Free Buffer List. 
This release process is entirely realized in the background 
procedure (157) to reduce to a minimum the number of 
instructions in the interrupt procedures. 

(162) The contents of the Output Queue (OQn) is tested: 
(165) If Output Queue (OQn) is empty, then it is set in a 

NON ACTIVE state (ST) in the Output Queue Table. 

(163) If Output Queue (OQn) is not empty, then the next 
Packet Pointer is dequeued (the HEAD value in the 
Queue Pointer is incremented) (166). The first Buffer 
Pointer of the Buffer List is dequeued (the HEAD value 
of the Packet Pointer is incremented) and saved (Cur- 
rent Buffer Pointer) in the Output Queue Table with the 
Packet Pointer (Current Packet Pointer). Once the Cur- 
rent Buffer Pointer (cur B_PTR) and the Current 
Packet Pointer (cur P_PTR) are identified, the Specific 
Purpose Processor (SPP) sends to the IOl a request to 
start the transmission of the Current Packet (START 
TRANSMISSION). 

2. Buffer Level (I01_EOB) 

FIG. 17 is a flow chart illustrating the interrupt procedure 
at the buffer level as represented in FIG. 14 (149). On the 
occurrence of an End Of Buffer interrupt (EOB): 

(170) The Current Packet Pointer (cur P_PTR) corre- 
sponding to the last Output Queue processed (OQn) is 
extracted from the Output Queue Table. 
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(171) The next Buffer Pointer (B_PTR) in the Buffer List 
is dequeued (the HEAD value of the Packet Pointer is 
incremented) This Buffer Pointer is first, set into the DMA 
Pointer (D_PTR1) attached to the 101 device and second, 
saved as Current Buffer Pointer (cur B_PTR) in the Output 
Queue Table. The Packet Pointer is also saved in the Output 
Queue Table as Current Packet Pointer. 

Release of Resources 


Data Transmission 
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FIG. 19 is a flow chart illustrating the release of the 
resources attached to the packets after their transmission as 
represented in FIG. 15 (157). This process is entirely located 
in the background procedure to minimize the number of 
instructions during the interrupts. 

(190) The contents of the Release Queue located in the 
Local Memory (131) is tested: 
If the Release Queue is empty, the release process is 
terminated and the background procedure can return in 
its initial waiting state (150). 
If the Release Queue is not empty then: 
(191) A Packet Pointer is dequeued from the Release 
Queue (the HEAD value of the Release Queue 
Pointer is incremented). 
(191) The Multicast Counter (MCC) is retrieved from 25 

the Buffer List Prefix. 
(151) The Multicast Counter (MCC) is decremented by 
one and tested (192): 

(193) If the Multicast Counter (MCC) value is equal 
to zero, then the transmission of the packet is 
terminated. All the resources attached to this 
packet must be released. The original Packet 
Pointer (P_PTR) (with the initial HEAD and 
TAIL values) previously saved is retrieved from 
the Buffer List Prefix. The Packet Pointer and 
Buffers Pointers are released respectively from the 
Free Packet List (FPL) and from the Free Buffer 
List (FBL). 

If the Multicast Counter (MCC) value is not equal to 
zero, then the multicast process is not terminated. 
(190) The contents of the Release Queue is tested one 
more time: 

If the Release Queue is not empty, the procedure goes 

on with a new Packet Pointer (191). 
If the Release Queue is empty, the release process is 

terminated and the background procedure can return 

in its initial waiting state (150). 


30 


35 


40 


45 


50 


FIG. 18 illustrates the data flow between the Specific 
Purpose Processor (SPP), the Direct Access Memory 
(DMA) and IOl device. First, the background procedure 
already described in FIG. 15 is in a waiting state. As soon as 
a packet is ready to be transmitted, the Packet Pointer is 55 
copied in the appropriate Output Queue n. This Output 
Queue n is set in an ACTIVE state and the SPP informs the 
101 device that it is ready to start the transmission on the 
output n (START TRANSMISSION n). At the reception of 
this message, the IOl responds to the DMA with a Service 60 
Request (SRI n) to request a first Buffer Pointer, the DMA 
triggers an End Of Buffer interrupt (I01_EOB n as shown 
in FIG. 17) to extract from the Output Queue Table the 
Current Buffer Pointer (cur B_PTR n) of the Output Queue 
n. The DMA orders simultaneously the Buffer Memory 65 
(BM) to present the buffer identified by said Current Buffer 
Pointer on the BMIO and the IOl to read this buffer (DS1 


n Data Service message). The process goes on with the next 
buffer until the end of the packet. At this time the DMA 
raises an End Of Packet interrupt Q01_EOP n as shown in 
FIG. 16) and informs the IOl device (EOP1 n). 

It is important to notice that all Output Queues are 
processed independently and in parallel under control of the 
IOl device according to the different protocols and speeds 
used on the output lines. 

We claim: 

1. A line adapter for a packet switching node in a 
communication network, including a programmable pro- 
cessing means (SPP) for receiving and transmitting data 
packets of fixed or variable length to one or more outputs, 
said line adapters comprising: 

a first storing means for buffering each data packet in one 
or more buffers; 

means for identifying said buffers, 

a second storing means including means for queuing said 
buffer identifiers in buffer lists, each buffer list identi- 
fying a data packet; 

means for identifying said buffer lists; 

means for queuing, in said second storing means, buffer 
list identifiers in packet lists, each packet list identify- 
ing a queue; 

means for identifying said packet lists; 

means for processing a routing header of each data 
packet; 

means for associating with each output an output queue 
for stacking the packet list identifiers of the data 
packets to transmit on said outputs; 

means for routing the data packets to one or a plurality of 
outputs; 

means for processing said routing header further com- 
prising means for determining and selecting the output 
queue corresponding to the destination of each data 
packet; 

said means for routing further comprising: 

means for copying the buffer list identifier of the data 

packet to transmit in the output queue corresponding to 

said selected output; 
means for handling each data request for said outputs in 

real time; 

means for releasing said buffers in said first storing 
means, and said buffer identifiers and said buffer list 
identifiers from said second storing means; 

said means for handling data requests further comprising 
means for handling independently the output queues, 
which further includes: 

means for dequeuing in parallel the packet identifiers 
requested by the outputs; and 

means for dequeuing the buffer identifiers stacked in the 
buffer list identified by said packet identifiers. 

2. The line adapter according to claim 1 wherein: 

said first storing means includes means for writing and 
reading said data packets in buffers of fixed length 
independently of said programmable processing 
means; and 

said second storing means includes means for separately 
storing said buffer lists and said packet lists under 
control of said programmable processing means. 

3. The line adapter according to claim 1 wherein said 
programmable processing means (SPP) comprises: 

a background procedure for processing the data packets, 
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a plurality of real time procedures triggered under control 
of each output to request the contents of a new buffer, 
said real time procedures being independent of the 
routing mode used. 

4. The line adapter according to claim 3 wherein said 
means for releasing in said means for routing is executed in 
said background procedure. 

5. The line adapter according to claim 1 wherein: 
said means for identifying said buffers includes buffer 

pointers (B_PTR) identifying said buffers and stacked 

in one or more buffer lists (B_UST); 
said means for identifying said buffer lists includes packet 

pointers (P_PTR) identifying said buffer lists 

(B_LIST) and stacked in one or more packet lists 

(P„LIST); and 
said means for identifying said packet lists include queue 

pointers (Q_PTR) identifying said packet lists 

(P_LIST) and stacked in one or more queue lists 

(Q_UST); and 
wherein each buffer list, packet list and queue list includes 

a prefix for storing information related to the data 

contained in said each buffer list, packet list and queue 

list. 

6. The line adapter according to claim 5 wherein said 
buffer list prefix includes control and routing information 
contained in the data packet header. 

7. The line adapter according to claim 6 wherein said 
buffer list prefix further includes a multicast counter (MCC 
COUNTER) and a packet pointer (P_PTR). 

8. The line adapter according to claim 7 wherein a status 
of each output queue is stored in an output queue table 
located in said second storing means, said status including: 

the output queue pointer (Q_PTR), 

a current packet pointer (cur P_PTR) for the last packet 35 
dequeued from the output queue; 

a current buffer pointer (cur B_PTR) for the last buffer 
dequeued from the current packet pointer, and 

a state of the output queue depending on if the output 
queue is active, an output queue being active during the 40 
lime said data packet identified by said current packet 
pointer is transmitted to said selected output. 

9. The line adapter according to claim 8 wherein said 
background procedure includes: 

means for detecting a new buffer list; 
means for reading the routing information in the buffer list 
prefix; 

means for initializing the multicast counter in the buffer 

list prefix with the number of outputs towards which 5 o 

the data packet must be transmitted; 
means for copying the packet pointer of the buffer list in 

its own prefix; 
means for selecting an output queue according to the 

destination of the rtetg packet, queuing in said output 

queue the packet pointer and testing the state of said 

output queue, wherein: 
if the state is active a new output queue is selected 

according to the other destinations of the data packet; 
if the state is not active: 
a packet pointer is dequeued from said output queue and 

saved as the current packet pointer in said output queue 

table; 

a buffer pointer is dequeued from the buffer list identified 65 
by said current packet pointer and saved as the current 
buffer pointer in said output queue table; 
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a message is sent to the output; 

the output queue is set in an active state; and 

a new output queue is selected according to the other 

destinations of the data packet; 
when said packet pointer has been queued in all selected 

output queues the background process returns to its 

waiting state. 

10. The line adapter according to claim 9 wherein said 
plurality of real time procedures comprise: 

a first real time procedure triggered by the outputs on 

request for a new buffer including: 
means for getting the packet pointer corresponding to the 

last output queue processed in the output queue table; 
means for dequeing the next buffer pointer in the buffer 

list identified by said packet pointer, 
means for saving said packet pointer into the output queue 

table as the current packet pointer; 

means for saving said buffer pointer into the output queue 

table as the current buffer pointer; 
a second real time procedure triggered by the outputs on 

request for a new packet including: 
means for getting the packet pointer corresponding to the 

last output queue processed in the output queue table; 
means for queuing said packet pointer in a release queue 

for a delayed release; ' 
means for testing if the output queue is empty or not, 

wherein: 

if said output queue is empty, said output queue is set in 

a non-active state; 
if said output queue is not empty: 
a next packet pointer is dequeued from said output queue; 
said next packet pointer is saved as the current packet 

pointer in the output queue table; 
a first buffer pointer of the buffer list identified by said 

packet pointer is saved as the current buffer pointer in 

the output queue table; and 
a message is sent to the output associated with said output 

queue. 

11. The line adapter according to claim 1 wherein the 
management of the buffers in said first storing means is 
realized by means of a first permanent list (free buffer list) 
containing all of the buffer pointers. 

12. The line adapter according to claim 1 wherein the 
management of the buffer lists in said second storing means 
is realized by means of a second permanent list (free packet 
list) containing all of the packet pointers. 

13. The line adapter according to claim 1 wherein said 
releasing means in said background procedure comprises: 

means for testing the contents of the release queue: 
if the release.queuc is empty, then terminating the release; 
if the release queue is not empty: 
dequeuing a packet pointer from the release queue; 
decrementing the buffer list prefix counter by one; 
testing if the multicast counter value is equal to zero, 
if the multicast counter value is not zero, then testing the 

contents of the release queue: 
if the release queue is empty, then terminating the release; 
if the release queue is not empty, then dequeuing the next 

packet pointer from the release queue; 

if the multicast counter value is zero: 
retrieving the original packet pointer from the buffer list 
prefix identified by said packet pointer; 
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releasing the buffer list and buffers identified by said 
packet pointer from the free packet list and the free 
buffer list; 

testing the contents of the release queue. 

14. The line adapter according to claim 5 wherein each list 3 
pointer (packet pointer (P_PTR) or queue pointer 
(Q_PTR)) comprises: 

a first field (LID) for identifying the list; 

a second field for identifying a next pointer (TAIL) to 1{} 

stack in said list; 
a third field for identifying a first pointer (HEAD) stacked 

in said list. 

15. The line adapter according to claim 14 wherein said 
means for queuing comprises: is 

means for incrementing the TALL field of the list pointer; • 
means for simultaneously storing the pointer identified by 

the TAIL field in the list identified by the LID field; 
means for generating a list full indicator. 

16. The line adapter according to claim 14 wherein said 20 
means for dequeuing comprises: 

means for incrementing the HEAD field of the list pointer; 
means for simultaneously reading the pointer identified by 

the HEAD field, in the list identified by the LID field; 25 
means for generating a list empty indicator. 

17. The line adapter according to claim 15 wherein said 
means for queuing further includes a means for testing said 
list full indicator, and said means for dequeuing further 
includes a means for testing said list empty indicator. 30 

18. The line adapter according to claim 1 wherein said 
programmable processing means includes: 

an arithmetical and logical unit, 

a register file, 3s 

a. sequencer, 

an instruction file, 

a direct memory access controller module, and 
a physical memory address generator. 

19. A routing method in a line adapter for a packet 40 
switching node in a communication network, including 
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programmable processing means (SPP) for receiving and 
transmitting data packets of fixed or variable length to one 
or more outputs, said routing method comprising the steps 
of: 

buffering each data packet in one or more buffers in a first 

storing means; 
identifying said buffers; 

queuing, in a second storing means, said buffer identifiers 
in buffer lists, each buffer list identifying a data packet; 
identifying said buffer lists; 

queuing buffer list identifiers in packet lists in said second 

storing means, each packet list identifying a queue; 
identifying said packet lists; 
processing a routing header of each data packet; 
associating with each output an output queue for stacking 

the packet list identifiers of the data packets to transmit " 

on said outputs; 
routing the data packets to one or a plurality of outputs; 
said processing the routing header step further comprising 

the step of determining and selecting the output queue 

corresponding to the destination of each data packet; 
said routing step further comprising the steps of: 
copying the buffer list identifier of the data packet to 

transmit in the output queue corresponding to said 

selected output; 
handling each data request for said outputs in real time; 
releasing said buffers in said first storing means, and said 

buffer identifiers and buffer list identifiers from said 

second storing means; 
said handling each data request step including the step of 

handling independently the output queues by 
dequeuing in parallel the packet identifiers requested by 

the outputs; and 
dequeuing the buffer identifiers stacked in the buffer list 

identified by said packet identifiers. 

***** 
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ABSTRACT 


A communication system includes a master station con- 
nected to a telephone line network and a plurality of remote 
stations for transmitting and receiving digital data to and 
from the master station through a radio transmission line. 
Among the master station and the remote stations, one 
channel is divided into a plurality of slots. One of the 
plurality of slots is assigned as a control slot to transmit and 
receive data in a time-division manner. The master station 
transmits control data to the remote stations by the control 
slot. The master station transmits control data by the control 
slot at every predetermined period and also transmits data 
concerning a communication by the control slot during a 
predetermined period. 

11 Claims, 6 Drawing Sheets 
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COMMUNICATION APPARATUS FOR TDMA According to Ibc circumstances, control data is received 

SYSTEM during the empty slot provided between the transmission slot 

BACKGROUND V3 and the reception slot VI* of the roaster station. The 

transmission of control data in the slot assigned to the 

1. Held of the Invention . 5 ^^1 slot C is not <^ed out for every fxarne. But, it is 

The present invention relates to a transmission and recep- customary that such transmission is earned out only once 

don apparatus and, more particularly, is directed to a trans- ^ a from several frames to several tens 

mission and reception apparatus for transmitting and recav- Q j frames. 

tag data based on a TDMA system . . ...... , x , .... 

% „. . , t™-*™ tf one channel is divided into slots as described above, 

2 Background of me Invention 10 toco communication data is transmitted and received among 

A communication system called a TDMA (time division oncm3Stcrsta£ion and the three remote stations by using one 

multiple access) system h^hoetofore been put into prac- (i fiin&Ic tranWion frequency). Therefore, it is 

heal use. According to the TDMA system, a communion ^ fof tfje ^ statioD to s^U^^y comrnuni- 

can be made by a slotarrangement such as shown m FIG^ ^ ^ dcviccs with fcwcr ch^^. 

1A to ID. In a radio telephone system based on the TDMA 15 . ^ . _ . . ^ , „ 

_ . . 1 , ' . , 13 Even thoueh the master station can be connected to a 

system, for example, a master station serving as a base t , " ~~_ ™; " , 

.T-.I-Z7... -i ..T-77T- .1 -i: i • 5 i. plurality of remote stations by using one channel in a 

station and a remote station serving as a terminal device can r . . / , . ' ^> ^ _ . . 

communicate with each other through a radio transmission raul ? lc ^fashion, the number of remote stetions that 

line. When the remote station and me master station com- f" * connected to &e master station in a multiple access 

mnnicate with each other via the radio transmission line, a - ^ C °V A S J" 1 *"*** sh0Wn ™ 

telephone communication can be made between the person 20 nGS ^ 1D ' ^JJ^i^ ^ * 

being called who is connected through the telephone line * * c ^ ^"fe* one channel at 

network to the master station and the remote station. In the ? B mc f 1 ? 15 a UnprOV - 

case of a communication system based on the TDMA m * the effiaency of using a transmission channel by increas- 

system, a single master station and a plurality of remote a '^^^ rf ^ f toon5 coimcCtC<1 to thc 

stations can communicate with one another by using one sUb0D through one channel. 

channel (single frequency). As shown in FIG. 1A, one . In order to increase the number of remote stations that can 

frame, which is a unit for making a communication, is 1x1 connected to the master station via one channel, it has 

divided into a plurality of slots (8 slots) on the master station bcen proposed that the number of slots provided in one 

side. In this case, one frame has a duration of 5 milliseconds 30 fr*™ 00 increased. In this case, however, if the number of 

and one slot has a duration of 0.625 millisecond. Commu- sl0 ^ increased, then either the duration of the frame 

nicatioc is carried out by repeating the units. period is extended or the duration of the slot period is 

When one frame is composed of 8 slots, up to three m V»*tter option, however, it is frequently 

remote stations a, b, c can be connected simultaneously to observed that audio data cannot be compressed satisfacto- 

the master station by using one channel. Specifically, the first 35 "v* 

slot of one frame is assigned to a control slot C during which SUMMARY OF THE INVENTION 
the master station transmits control data to the remote 

stations. The control data is used to carry out a control, such 11 ** therefore an object of the present invention to provide 
as assigning slots for transmitting communication data and a transmission and reception apparatus in which the above- 
individually accessing thc remote stations. Accordingly, the 40 meationcd Problem can be solved, 
control data (hat Is transmitted from the master station It is another object of the present invention to provide a 
during the control slot C is received by the remote station communication system in which the above-mentioned prob- 
which communicates with the master station, i.e., any one of lem can be solved. 

the remote stations a, b, c shown in FIGS. IB, 1C and ID. According to the present invention, there is provided a 

As shown in FIGS. IB, 1C and ID, the three remote 45 transmission and reception apparatns. The transmission and 

stations a, b, c are simultaneously connected to the master reception apparatus divides one channel into a plurality of 

station. Three slots VI, V2, V3 which follow the control slot slots. One of the plurality of slots is assigned as a control slot 

C are slot periods during which communication data is in which control data is processed in a time division nml- 

transmitted from thc master station to the three remote tiplex fashion and transmitted and received together with 

stations a, b, c. The communication data are results from 50 data concerning a communication. The transmission and 

converting an audio signal of 5 milliseconds into digital data reception apparatus includes a transmission and reception 

is compressed so that the digital data of 5 milliseconds can unit and a control unit The transmission and reception unit 

be transmitted during one slot period. The three remote processes data in a time-division multiplex fashion and 

stations a, b, c cany out receptions Ra, Rb, Rc during transmits the data thus processed. Also, the transmission and 

corresponding slot periods as shown in FIGS. IB, 1C and 55 reception unit receives transmitted data. The control unit 

ID. controls an operation of the transmission and reception unit 

Then, the next one slot after slot V3 is not used Le. it is, The control unit transmits data concerning a communication 

an empty slot The master station receives communication by the control slot at a time to transmit control data during 

data from the three remote stations a, b, c during the last a predetermined period provided after the control data has 

three slots VI', V2\ V3'. Accordingly, the three remote 60 been transmitted by the control slot 

stations, a, b, c carry out transmissions Ta, Tb, Tc of According to the present invention, there is provided a 

communication data during corresponding slot periods as communication system which includes a master station 

shown in FIGS. IB, 1C and ID. Also in this case, commu- connected to a telephone tine network and a plurality of 

ni cation data are results from converting an audio signal of remote stations which transmit and receive digital data to 

S milliseconds into digital data that is compressed so that the 65 and from the master station via a radio transmission line, 

digital data of 5 rnilliseconds can be transmitted during one Among the master station and the remote stations, one 

slot period. channel is divided into a plurality of slots and one of the 
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plurality of slots is assigned as a control slot during which 
digital data is transmitted and received in a time division 
manner. Also, control data is transmitted from the master 
station to the remote stations by the control slot Hie 
master-station transmits control data by the control slot at 
every predetermined period also transmits data concerning a 
communication by the control slot during a predetermined 
period. 

According to the present invention, data concerning a 
communication is transmitted by using the control slot when 
control data is not transmitted by the control slot Thus, it is 
possible to increase the number of remote stations that can 
be connected to the master station by using one channel. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIGS. 1A through ID are timing charts showing an 
example of an arrangement of slots based on the TDMA 
system; 

FIG. 2 is a block diagram showing an arrangement of a 
master station according to an embodiment of the present 
invention; 

FIG. 3 is a block diagram showing an arrangement of a 
remote station according to the embodiment of the present 
invention; 

FIG. 4 is a flowchart to which references will be made in 
explaining the condition of when an inventive slot is in use; 

FIGS. 5A through 5E are timing charts showing an 
arrangement of slots provided when it is detected that 


conversion processing to convert intermittent received data 
extracted from respective slots to consecutive data. 
Moreover, each of the speed converting units 4a through An 
carries out a speed conversion processing to convert trans- 
mission data to intermittent data so that the transmission 
data can be inserted into the respective slots. Network 
interfaces 5a, Sb, . .... 5n arc connected to the speed 
converting units 4a through 4n, respectively. Each of the 
network interfaces So, 5b, . . . , Snis connected to a digital 
telephone line network, such as an ISDN (integrated service 
digital network) line network. The network interfaces 5a 
through Sn convert a dock rate in order to convert data 
output thereto from the speed converting units 4a through 4n 
to data transmitted to the telephone line network. Moreover, 
the network interfaces 5a through Sn convert the clock rate 
in order to convert data received from the telephone line 
network to radio transmission data by the radio interface 2. 

The processing in each circuit provided within the master 
station shown in FIG. 2 is executed at a synchronized timing 
under the control of a control unit 6. 

Data supplied to the speed converting units 4a through 4n 
arc also supplied to a mute detecting unit 7. The mute 
detecting unit 7 (letermines whether ox not communication 
data transmitted from the telephone line network side to the 
network interfaces So to 5n at every network are placed in 
the mute state. As a method used by the mute detecting unit 
7 to determine whether or not communication data is placed 
in the mute state t there are enumerated a method of detecting 
_ a mute state code added when a transmitted audio signal is 

communication data transmitted by a slot V4 is placed in the 30 placed in the mute state and a method of detecting based on 


10 


15 


20 


23 


mute state according to the embodiment of the present 
invention; and 

FIGS. 6A through 6E arc timing charts showing an 
arrangement of slots provided when it is detected mat 
communication data transmitted by the slot V4 is not placed 
in the mute state according to the embodiment of the present 
invention. 
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the state of transmitted audio data whether or not commu- 
nication data is placed in the mute state. In mis case, when 
the four speed converting units 4a through An are all in use, 
or four networks are connected to the master station, the 
mute detecting unit 7 detects the mute state of the network 
connected last When the mute detecting unit 7 detects the 
mute state, a detected result of the mute detecting unit 7 is 
supplied to the control unit 6. 

The control unit 6 supplies the control data to the multi- 
plexing unit 3 at a predetermined time to enable the control 
data to be transmitted to the remote station side by a 
predetermined slot Also, control data of up-link which is 
transmitted from the remote station by a predetermined slot 
and which is supplied to the nanltjplexing unit 3 is supplied 
to the control unit <S. 

The control unit 6 carries out a communication control 
processing such mat up to four remote stations , Le., four line 
networks can communicate with one another simultaneously 
by one channel through radio waves. When the four line 
networks are simultaneously In use, if it Is determined by the 
mute detecting unit 7 that communication data transmitted 
from the telephone line network side to the speed converting 
unit An of one line network is placed in the mute state, then 
the control unit 6 supplies control data, which is to be 
transmitted to the remote station, to the multiplexing unit 3. 
However, when a transmission of control data is stopped 
during a predetermined period, even if it is determined by 
the mute detecting unit 7 that communication data is not 
placed in the mute state, then the control unit 6 supplies the 
control data to the multiplexing unit 3 to transmit the same 
by a predetermined slot This processing will be described 
later on. 

FIG. 3 shows in block form an arrangement of a remote 
can be connected to the multiplexing unit 3 is selected to be 65 station according to the embodiment of the present inven- 
4 so that there are prepared 4 line networks. Each of the tion. As shown in FIG. 3, an antenna 11 is connected to a 
speed converting units 4a through An carries out a speed radio interface 12. The radio interface 12 demodulates or 


DESCRIPTION OF THE INVENTION 

A transmission and reception apparatus according to an 
embodiment of the present invention will hereinafter be 
described in detail with reference to the drawings. 

In this embodiment, the transmission and reception appa- 
ratus according to the present invention is applied to a 4J 
radiotelephone system in which a signal, which was con- 
verted to digital data, is transmitted and received among a 
master station serving as a base station and terminal stations 
serving as remote stations in a TDMA system fashion. FIG. 
2 shows in block form an arrangement of an inventive 
master station. 

As shown in FIG. 2, an antenna 1 is connected to a radio 
interface 2. The radio interface 2 demodulates a received 
signal received at the antenna 1 or modulates the received 
signal so that the signal can be transmitted or received 
satisfactorily. This radio interface 2 is connected to a mul- 
tiplexing unit 3. The multiplexing unit 3 separates data 
inserted into respective slots of the received signal and 
synthesizes data so that data is inserted into the respective 
slots of a transmitted signal. An arrangement of the slots will 
be described later on. 

A plurality of speed converting units 4a, 4b, . . . , An are 
connected to the multiplexing unit 3. In this embodiment, 
the number of the speed converting units 4a through An that 
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modulates a received signal received at the antenna 11 so network that is in use, as shown in FIGS. 6A to 6E, it is 

that the signal can be transmitted or received satisfactorily frequently observed that this control slot C becomes a 

upon radio commu nlcatioD . The radio interface 12 is con- transmission slot V4 wherein control data is transmitted 

nected to a transmission and reception timing control unit fro™ the master station to the remote stations. When this 

13 Hie transmission and reception control unit 13 extracts 5 transmission slot V4 is in use, as shown in FIGS, 5A to 5E 

only a signal of a predetermined slot from a received signal ** d nGS - 6A to 6E, the first slot of the 4 slots of the second 

and carries out a necessary processing based on the TDMA **** of 0QC « employed as a reception slot V4* 

system in which a transmission signal is transmitted at a corresponding to the ttansmfasion slot V4. 

timing of a predetermined slot. The transmission and recep- Conditions by which the slots are set as described above 

tion timing control unit 13 controls the timing under the 10 ^ described below. Initially, when mere are three 

control of a control unit 17 which controls the cemmunica- rciDOtc statioils arc connected to the master station 

tion of the remote station. In this case, the control unit 17 (three remote stations win hereinafter be referred to as 

carries out a control operation based on control data 'demote stations Ml, M2 M3'), transmission and reception 

extracted from the received data by the transmission and 816 earned out among the remote stations Ml, M2, M3 

reception timing control unit 13, Lc, control data transmit- u shown in FIGS. 5B, 5C, 5D and FIGS. 6B, 6C, 6D by using 

ted thereto from the master station. When the transmission transmission slots VI, V2, V3 and the reception slots VI', 

and reception tuning control unit 13 cannot receive control V2 '> yy ' m nGS - 5 A and 6A. In mat case, the master 

data, the transmission and reception timing control unit 13 tansinits contro1 data to the remote stations by using 

memorizes control data received last and continues carrying mc contro1 dots C of the starting portions of the respective 

out a control operation based on the control data thus stored 20 frames to control the communication states of the respective 

therein, remote stations Ml, M2, M3 on the basis of the control data. 

The transmission and reception timing control unit 13 is ^L^^^Z^rT "* * ^ 

connected to an audio MODEM (modulator and '""T^.™?" £ £ \ 

demodulator) unit 14. The audio MODEM unit 14 carries According to this embodiment, in the communication 

out a demodulation processing wherein digital audio data is 25 state with this ^ structure, another remote station M4 

extracted from the received data and converted into an ^"S?^ the master station ( L e. ( four remote 

analog audio signal and a processing wherein the analog ^tions Ml, M^M3 M4 in total can be connected to the 

audio signal is converted to digital audio data and transmit- ™«» Nation). When the four remote stations Ml M2, M3, 

ted as a transmission signal. A speaker 15 and a microphone ?*4 are connected to the master station, the control data may 

(MIC) 16 arc connected to the audio MODEM unit 14. The » be transmitted by using the control slot C or the control slot 

audio signal that has been demodulated by the audio C ""V b ' uscd ™ the n-ansmission slot V4 through wMch 

MODEM unit 14 is emanated from the speaker IS and a control 15 traimnittcd from mc staUoD to *» 

sound picked up by the microphoae 16 is supplied to fee remote station M4 This decision will be described below 

audio MODEM unit 14 as an audio signal with reference to the flowchart shown in FIG. 4. 

, . „. . . » ^ 1. j- 35 As shown in FIG. 4, following the start of operation, it is 

Therefore, in the inventive remote station, when audio . . . . • • „ rt „ 7^ „, k -k— «■ « wL,-™ 

date supplied from the transmission and reception timing m ^sion step 101 whether or not a ormng 

contrSXn to the audio MODEM unit ^interrupted l *°° m f S a ^f? 1 d^traiismission timing. In other words, 

~~ 7 " , \ . / . 7l . according to the inventive communication system, since it is 

Kmporcnly phced in the nmte ^ unol the ousmkaon itljdeten ^ tod ^ 11 ^ 1Mwh ^ er ^; otatJlnillg 

and reception timing control unit 13 starts supplying audio , . . . . t ... , . 

datotothcaudioMODEMunitM. becomes a transn^ssiOD bmmg at which the control slot C is 

mwa " 1 umt xt. assigned once per 20 frames. If a timing is not the .trans- 

Akey 18 such as a dial key is connected to the control unit ^ of ^ slot c w rcprcscntcd by a N0 

17 which carries out communication control in this remote ^ ^osloa step 101, then the processing proceeds to step 

station. The transmission and reception apparatus can make 4J 105, whereat the control slot C is used as the transmission 

an outgoing call or receive an incoming call as a telephone slo( v4 for me station M4 to transmit communication 

set when the key 18 is operated. data. Moreover, the remote station M4 shown in FIG. 6E 

A communication state among the inventive master sta- receives communication data by a reception slot R4. At that 

tion and the inventive remote stations will be described time, the reception slot V4' corresponding to the transmis- 

bclow with reference to a flowchart of FIG. 4 and commu- 50 8 i on slot V4 also is set and a transmission of data from the 

ni cation timing charts shown in FIGS. SAto SB and FIGS. remote station M4 by using the transmission slot T4 is 

6A to 6EL executed at this timing . 

Initially, a slot arrangement applied to the communication If on the other hand the timing becomes the control data 

according to this embodiment will be described below. transmission timing as represented by a YES at decision step 

According to this embodiment, as shown in FIGS. 5A to 5E 55 101, then the processing proceeds to the next decision step 

or FIGS. 6A to 6E, one frame is composed of 8 slots. 102. It is determined in decision step 102 whether or not 

Fundamentally, 4 slots of the first half of one frame are digital data transmitted from the telephone line Detwork side 

assigned to transmission slots of the master station and 4 of the network connected last is placed in the mute state. At 

slots of the second half of one frame are assigned to fee that time, it is determined on the basis of detected data from 

reception slots of the master station. This frame structure is 60 the mute detecting unit 7 whether or not the digital data is 

repeated and one frame has a duration of 5 milliseconds. placed in the mute state. If the digital data is placed in fee 

As shown in FIGS. 5A through 5B, the first one slot of one mute state as represented by a YES at decision step 102, then 

frame is assigned to the control slot C wherein the master the processing proceeds to step 104, whereat control data is 

station transmits control data to the remote stations when transmitted as the control slot C The frame structure pro- 

predetermined conditions arc satisfied, eg., four remote 63 vided at that time becomes such one as shown in FIGS. 5A 

stations are connected to the master station as will be through 5E. The control data used at that time is data mat is 

described later on. Depending on fee condition of the transmitted to all of the remote stations Ml to M4. 
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If on the other hand the transmitted digital data is not 
placed in the mute state as represented by a NO at decision 
step 102, then the processing proceeds to the next decision 
step 103. In decision step 103, it is determined whether or 
not N frames (eg., 40 frames) are passed since the control 
data was transmitted last The N frames have a period 
corresponding to a limit time wherein (he remote stations 
Ml to M4 can maintain a communication among them and 
the master station on the basis of the previously-received 
control data, Le„ the remote stations Ml to M4 can maintain 
a frame synchronization. Therefore, if control data were not 
transmitted once at least in the N frames, the remote stations 
could not communicate with the master station. 

If the N frames are passed as represented by a YES at 
decision step 103, then the processing proceeds to step 104, 
whereat the control data is transmitted as the control slot C. 
If on the other hand the N frames are not passed as 
represented by a NO at decision step 103, then the process- 
ing proceeds to step 105, whereat the control slot C is set to 
the transmission slot V4 of the remote station M4 to transmit 
communication data. 

In any case, a slot provided between the transmission slot 
V3 and the reception slot VI' of the master station is set to 
the reception slot V4' from the remote station M4 shown in 
FIGS. 5E and 6E to receive communication data transmitted 
by the transmission slot T4 of the remote station M4. 

The state shown at the timing t„ shown in FIGS. 5A 
through 5E shows the state that the control data is transmit- 
ted by the control slot C even when the digital data is not 
placed in the route state after it was determined in decision 
step 103 that the timing becomes the timing having the 
period corresponding to the limit time in which a commu- 
nication can be i"«int»iin-«i at mis riming 

As described above, according to the inventive frame 
structure, the master station can communicate with one 
remote station by using the slot assigned to transmit control 
data. Thus, it is possible to increase the number of the remote 
stations that can be connected to the master station by one 
without varying the frame period and the like. Id this case, 
if the control data is transmitted when the communication 
data transmitted to the remote station which uses this slot is 
placed in the mute state, then the control data is transmitted 
so long as no trouble occurs even though the transmission of 
data of one slot is failed during communication data is 
transmitted between the master station and the remote 
station. Therefore, it is possible to efficiently transmit both 
the communication data concerning audio data and the 
control data by using the control slot Incidentally, since the 
mute state of communication data appears frequently upon 
ordinary conversation, there is then the large possibility that 
control data can be transmitted at the predetermined period 

If the communication data is not placed in the mute state, 
then when the timing becomes the limit of time whereat the 
communication can be maintained by the control data trans- 
mitted last, control data is forced to be transmitted. 
Therefore, the communication states among the remote 
stations can be maintained satisfactorily. In this case, 
although communication data concerning audio data is inter- 
rupted once per N frames during one frame period in the 
remote station M4 which was connected to the master 
station last, in the remote station according to this 
embodiment, an audio signal reproduced at that time is 
placed in the mute state only during one frame period, Le., 
5 milliseconds. This one frame period is not long enough for 
the user to identify the mute state in actual practice. 
Therefore, mis one frame period does not hinder the user 
from hearing communication data substantially. 
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When control data or communication data is transmitted 
by one slot, since corresponding identification codes arc 
added to the control data and the communication data, even 
if the two data of the control data and the communication 

5 data are provided in the mixed state like this embodiment, it 
is possible to easily discriminate the communication data 
and the control data one from another by the remote station. 

While the four remote stations Ml to M4 are connected to 
the master station by using one transmission channel as 

[Q described above, when any one of the slots becomes dis- 
abled in use after a comrnunication done by the remote 
station which uses any of the slots VI and VI', V2 and V2\ 
V3 and V3' was finished, it is preferable that a communi- 
cation which uses the slots V4, V4' is moved to the disabled 
slot. If the slot in use is moved as described above, then it 

1 is possible to prevent communication data, i.e., a sound 
output from the speaker 15 from being interrupted tempo- 
rarily. 

The used slot can be moved by transmitting coxrespond- 
ing control data from the master station to the remote station 
on the basis of the control done by the control unit 6 of the 
master station. 

When control data such as a connection request signal is 
transmitted from the respective remote stations to the master 

23 station by using the same channel, il is sufficient to use the 
first slot V4' of the second half slots of one frame, for 
example. In this case, since the connection request signal 
need not be transmitted when a connection between the 
remote station and the master station is completed, the 

30 master station need not receive control data from the remote 
station under the condition that the four remote stations are 
all connected to the master station. Therefore, it is sufficient 
that the master station receives only communication data 
from the remote station under the condition that the control 

35 slot is used as the slot V4\ Thus, the four remote stations and 
the master station can communicate with one another satis- 
factorily. 

Further, while one frame is composed of 8 slots to 
multiplex four telephone networks as described above, the 
40 present invention is not limited thereto. It is needless to say 
that the control slot may be used as a communication slot 
when one frame is composed of several slots other than 8 
slots. 

Furthermore, while a transmission signal modulation sys- 
45 tern between the master station and the remote station and 
the like have not been described so far in detail, it is needless 
to say that the present invention can be applied to a variety 
of communication systems which transmit and receive data 
of slot arrangement 
50 Having described a preferred erntodiment of the inven- 
tion with reference to the accompanying drawings, it is to be 
understood that the invention is not limited to that precise 
embodiment and that various changes and modifications 
could be effected therein by one skilled in the art without 
55 departing from the spirit or scope of the invention as defined 
in the appended claims. 
What is claimed is: 

1. A transmission and reception apparatus including a 
channel divided into a plurality of frames, each of said 

60 plurality of frames being subdivided into a plurality of slots, 
one slot of each of said plurality of frames being assigned as 
a dual-purpose slot for carrying control data or communi- 
cation data and the remaining slots of each of said plurality 
of frames being assigned as communication slots for carry- 

65 ing communication data, wherein control data is transmitted 
and received together with communication data, said appa- 
ratus comprising: 
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transmitting and receiving means for transmitting data in 
the form of time-di vision-multiplexed data and for 
receiving data transmitted thereto; 

control means for controlling an operation of said trans- 
mitting and receiving means, said control means con- 
trolling said transmitting and receiving means to trans- 
mit said communication data in said dual-purpose slot 
of all subsequent ones of said plurality of frames at a 
time for transmitting said control data during a prede- 
termined period after said control data is transmitted in 
said dual-purpose slot of a previously transmitted one 
of said plurality of frames, said control means control- 
ling said transmitting and receiving means to transmit 
said control data in said dual-purpose slot when not 
transmitting said communication data, and said control 
means controlling said transmitting and receiving 
means to transmit said communication data in said 
communication slots in all of said plurality of frames, 

whereby each of said plurality of frames includes a 
dual-purpose slot carrying one of control data and 
communication data; and 

detecting means for detecting a mute state from an output 


10 


15 


20 


trolling said transmitting and receiving means to trans* 
mit said control data in said dual-purpose slot of first 
predetermined ones of said plurality of frames during a 
first predetermined period and said control means con- 
trolling said transmitting and receiving means to trans- 
mit said communication data in said dual-purpose slot 
of second predetermined ones of said plurality of 
frames during a second predetermined period, whereby 

■ the first predetermined ones of the plurality of frames 
and the second predetermined ones of the plurality of 
frames include all of the plurality of frames; and 

detecting means for detecting a mute state from an output 
signal supplied thereto by said transmitting and receiv- 
ing means, 

wherein said control means controls said transmitting and 
receiving means to transmit said control data in said 
dual-purpose slot of all subsequent ooes of said plu- 
rality frames while said detecting means detects said 
mute state. 

6. A transmission and reception apparatus according to 
claim 5, wherein said control means transmits said control 
data in a dual-purpose slot of one of said plurality of frames 


. , . J within a predctermLDed period after said control data is last 

signal supplied thereto by said transmitting and receiv- 25 -~ j - . , ^_ . . , 

transmitted in a previously transmitted one of said plurality 

— of frames. 

7. A transmission and reception apparatus according to 
claim 6, wherein said control means mutes an output audio 

30 signal when said control means detects that audio data 
supplied thereto from said transmitting and receiving means 
has been interrupted. 

8. A communication system comprising: 


ing means, 

wherein said control means controls said transmitting and 
receiving means to transmit said control in said dual- 
purpose slot of all subsequent ones of said plurality of 
frames while said detecting detects said mute 

state. 

2. A transmission and reception apparatus according to 
claim 1, wherein said control means transmits said control 
data In a dual-purpose slot of one of said plurality of frames 35 
within a predetermined period after said control data Is last 
transmitted in a previously transmitted one of said plurality 

of frames. 

3. A transmission and reception apparatus according to 
claim 1, wherein said control means transmits said control 
data in a dual-purpose slot of one of said plurality of frames 
after a predetermined time has elapsed from when said 
control data is last transmitted in a previously transmitted 
one of said plurality of frames. 

4. A transmission and reception apparatus according to 
claim 3, wherein said control means mutes an output audio 
signal when said control means detects that audio data 
supplied thereto from said transmitting and receiving means 
has been interrupted. 

5. A transmission and reception apparatus including a 
channel divided into a plurality of frames, each of said 
frames being subdivided into a plurality of slots, one slot of 
each of said plurality of frames being assigned as a dual- 55 
purpose slot far carrying control data or communication data 
and the re mainin g slots of each of said plurality of frames 
being assigned as communication slots for carrying com- 
munication data, wherein control data is transmitted and 
received together with communication data, said apparatus 60 

transmitting and receiving means for transmitting data in 
the form of tinie-di vision-multiplexed data and for 
receiving data transmitted thereto; 
' control means for controlling an operation of said trans- 
mitting and receiving means, said control means con- 


40 


45 


50 


65 


a master station connected to a telephone line network; 
and 

a plurality of remote stations for transmitting and receiv- 
ing digital data to and from said master station through 
a radio transmission line, wherein 

a communication channel between said master station and 
said remote stations is divided into a plurality of 
frames, each of said plurality of frames being subdi- 
vided into a plurality of slots, 

one slot of each of said plurality of frames is assigned as 
a dual-purpose slot carrying control data or communi- 
cation data and die remaining slots of each of said 
plurality of frames being assigned as communication 
slots for carrying communication data, 

said master station transmits control data to said remote 
stations in said dual-purpose slot of each of first pre- 
determined ooes of said plurality of frames dining a 
first predetermined period, 

said rn**t rr station transmits communication data in said 
dual-purpose slot of each of second predetermined ones 
of said plurality of frames during a second predeter- 
mined period, 

whereby the first predetermined ones of the plurality of 
frames and the second predetermined ones of the 
plurality of frames include all of the plurality of frames; 
and 

detecting means for detecting a mute state from on output 
signal of one of said plurality of remote stations, 

wherein said master station transmits said control data in. 
said dual-purpose slot of all subsequent ones of said 
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plurality of frames while said detecting means detects 

said mute state. 
9. A communication system according to claim 8, wherein 
a remote station communicates with said master station 
based on a most recently received control data when said 
remote station does not receive control data subsequent to 
said most recently received control data from said master 
station. 

1#. A communication system according to claim 9, 
wherein said master station transmits said control data in a 
dual-purpose slot of one of said plurality of frames after a 


10 


12 


predctermioed period has elapsed from when said control 
data was last transmitted in a previously transmitted one of 
said plurality of frames. 

11. A communication system according to claim 10, 
wherein said remote station mutes an output of said master 
station during a predetermined period when said remote 
station detects that audio data supplied from said master 
station has been interrupted. 
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ABSTRACT 


A node in a synchronous communication network performs 
communication by sending and receiving frames formed by 
successively adding pointers showing header positions to a 
plurality of data, wherein a comparison is made between the 
values of pointers added to data received from an opposing 
node side and the values of pointers to be added to me data 
sent from a home node to detect if the two values coincide. 
Synchronization has been established between the opposing 
node and the home node when they do. 

6 Claims, 13 Drawing Sheets 
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NODE PROVIDED WITH FACILITY FOR FIG. 4 is a view of mapping from VC-4 to STM-1; 

CHECKING ESTABLISHMENT OF FIG. 5A is a view of the configuration of a node 

SYNCHRONIZATION (transmission apparatus) in a synchronous communication 

network to which the principle of the present invention is 

This is a continuation of application Ser. No. 08/213254. 3 applied; 

filed Mar. 15. 1994 abandoned FIG. SB is a view of the signal formats of a received 

„ ™™«,r™ ^ mrr , ™ signal Y and transmission signal V in FIG. 5A; 

BACKGROUND OF THE INVENTION „ . c * M . c ' 

FIG. 6 is a view of an embodiment of the present 

1. Field of the Invention 10 invention; 

The present invention relates to a system for checking for FIG. 7 is a view of an example of the configuration of a 

the establishment of synchronization in a synchronous com- demultiplexer unit in the present invention; 

munication network, more particularly relates to a system FIG. 8 is a view of an example of the configuration of a 

for checking for the establishment of synchronization which multiplexer unit in the present invention; 

confirms the synchroiuzation from the values of pointers of 15 nG 9 a ^ ^ k of ^ of ^ 

a received signal and transmission signal. present invention; 

In recent yearsjlong with the standardization of digital nG 10 * a view of a more specific example of the 

communication netwe^jynchronous mulup exing ; has conf ^ uration of me Dodc 10 in nG . 6; 

spread even to the higher order region. As a result of this, it "** . 

has become important to confirm the synchronization 20 ^ G U 15 a ™ °f * m *fL!!*^ c CIamptC of mc 

between transmission apparatuses in a synchronous com- configuration of the node 30 in FIG. 6; 

munication network, such as SONET in the U.S. FIG. 12 is a view of a more specific example of the node 

2. Description of the Related An 20 in FIG. 6; and 

In conventional communication systems, a synchronous ™- 13 is a of an e5iam P te of a <kn»"Wplex Unction 

network is formed only at the tower order side. Since the 23 umt showo in FIG. 7 and FIG. 12. 

speed is slow, asynchronization between transmission appa- nFQrcnmflM flF THP PRFFFRRFD 

ramses does not pose that much of a problem. Therefore DESCR™ OFT^F^D 
there is no need to check the state of synchronization and 

accordingly there has been no continuous confirmation of Before describing the embodiments of the present 

synchronization performed. . M invention, the related art and the problems therein will be 

Even in a synchronous communication network covered first described with reference to the related figures, 
by the present invention, which will be explained later with p^st, an explanation will be made of a known synchro- 
reference to the drawings, there has been no general check nous communication network to which the present invention 
of the establishment of synchronization including the higher 35 is applied referring to FIG. 1 to FIG. 4. 
order groups. pj G 1 ^ a v j ew illustrating a multiplex hierarchy in a 

As explained above, if synchronous multiplexing is per- synchronous communication network, 

formed up to the higher order region due to the standard- b nQ j vc . n ^ a ^sic virtual container. TU-12 is a 

ization of digital ccminunication networks in recent years. unit , TUG-21 and TUG-32 are tributary unit 

asynchronization among transmission apparatuses some- ^ g^ps. vC^I is a higher order virtual container. AIM is an 

times causes trouble. Accordingly, the problem has arisen administrative unit, and STM-1 is a synchronous transfer 

mat it is not possible to check for the establishment of module 

synchronization by a simple method or apparatus. _ . ' , ..... . - . . 

3 The tributary unit TU-12 is formed by adding to the basic 

SUMMARY OF THE INVENTION virtual container VC-12. formed from a 2.048 Mb/s coo- 

45 tainer C-12. a path overhead VC-12 POH for transmitting 

Accordingly, the present invention, in view of the above- control information among transmission apparatuses. By 

mentioned problems, has as its object to check the estab- adding to the tributary unit TU-12 a pointer TU-12 PTR and 

lishmentof synchronization by a simple means at each node. multiplexing these in triplicate, the tributary unit group 

To attain the above object, the present invention takes TUG-21 is formed, 

note of the values of the pointers added to the data of the 50 By multiplexing seven tributary unit groups TUG-21. the 

signal received from the opposing node side and the values tributary unit group TUG-32 is formed. Further, by multi- 

of the pointers added to the data when a signal is sent out plexing the tributary unit groups TUG-32 in triplicate and 

from the home node as a transmission signal and judges that adding the path overhead VC-4 POH. the higher order 

synchronization between the two nodes has been established virtual container VC-4 is formed. 

when the values of the pointers match. » By addbg to the ^d* virtual container VC^t the 

BRIEF DESCRIPTION OF THE DRAWINGS P 3 * overhead VC-4 POH. the adrriinistrative unit AU-4 is 

farmed By adding to the administrative unit AU-4 the 
The above object and features of the present invention pointer AU-4 PTR. the synchronous transfer module STM-1 
will be more apparent from the following description of the ^ is formed. 

preferred embodiments with reference to the accompanying The abo VC multiplex hierarchy is a basic item in the 
drawings, wherein: CCTTT RccommendaUons. 

nG. 1 is a view illustrating a multiplex hierarchy in a mG 2 is a view of the overhead boundary in a synchro- 
synchronous coinmunication network; nous communication network. In FIG. 2. a path overhead 

FIG. 2 is a view of the overhead boundary in a synchro- « VC-12 POH for transmitting control information between 
nous communication network; transmission apparatuses is added between the basic virtual 

FIG. 3 is a view of mapping from VC-1 to VC-4; containers VC-12. a path overhead VC-4 POH for transitu t- 
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ting control information between transmission Apparatuses More specifically, the method of the present invention 
is added between the higher order virtual containers VC-4. a ) demultiplexes the signal Y received from the opposing 
and a section overhead STM-N MSOH for transmitting oode ^ me ^ x ^ pointers (a) by the timing of the 
control information between transmission apparatuses and a received signal Y and holds the data X. 
section overhead STM-N RSOH for transmitting control 5 . ^ . . , 4 v . A . . ,„ 
information between repeaters are added between the syn- , b > successively reads out the held data X by the dock (f 
cfaronous transfer nwdulesSTM-N. of * c c,odc sourc * 3 in the k<>vac node, adds pointers (a) 
The N in the above-mentioned "STM-hT stands for I. 2. showin S thc ^ *<= same to the data X. and muJti- 
3. etc. In FIG. 2. the example is shown where N=l. Further. P lCTes thcm t0 farm thc transmission signal Y\ 
MSOH means a multiplexer SOH, while RSOH means a lQ c) compares the pointers (a) demultiplexed from the 
regenerator SOH. Note that RSOH is. for example, an received signal Y and the pointers (p) added to the trans- 
order- wire signal which can be monitored by the repeaters in mission signal Y\ and 

the figure. MSOH cannot be monitored by these repeaters d ) detects that the dock of thc transmission node of the 

and can only be monitored by the STMs at the two ends. rcccivco sigml Y and the dock of the home node are 

Incidentally, the signals C-12 are low order level signals synchronized when the two pointers arc compared and found 

used at the subscriber side. ^ matrh 

FIG. 3 is a view of mapping from VC-1 to VC-4. In FIG. _ , , , . . tl . , . ,- lk „ TVW 

3. the tributary unit groupTUG 21 is formed by adding to n ^4 1 * ! ^multmlexer unit (DMUX) 
a tributary ^oit 12. forrned by adding to a basic virLu whi <* ou the data X from the received ^nalYand 
containerVC-12 formed from a 1048 Mb/s container C-12 monttors *% valucfi a of addcd to « * 
a path overhead POH. VI as the pointer TU-12 FTR and 20 f* ^ ^f^^f^ 2 15 fff 100 ^ the 
„^«i«; nn :„ data X demultiplexed at the demultiplexer unit 1. Reference 
multiplexing these in triplicate. ...... . . . ,, . . 

« . . . • . numeral 3 is a clock source which supplies a master dock 

By multiplexing ft**? umt grams TUG-21, fte of mc ^ (transmission apparatus^ Reference numeral 

tributary unit group TUG-32 is formed ^Further, by mum- < . % ^plexei unit (MUX)lJruch multiplexes the data X 

plexing the tributary unit groups 1J JO-M in tiipticate and ^ ^ ^ daaamtmd ^ 

transmission 

adding the path overhead VC-4 POH. me higher order ^ ^ ^ a ^ ^ coofirms the headers of the 

virtual container VC^ is formed. data X. adds pointers (p) showing the positions of thc same. 

FIG. 4 is a view of mapping from VC-4 to STM-1. In FIG. Md coatmuously monitors the values B of the pointers at that 

4, an administrative unit AIM is formed by addingto the ^ Reference numeral 5 is a comparing unit which 
higher order virtual container VC-4 a pointer AU-4 PTR and x compares ^ values a of me poiniers monitored at the 
a synchronous transfer module STM-1 is formed by adding denuUtiplexcr unit 1 and the values 6 of the pointers 
to the administrative unit AU-4 the section overheads monitored & tj, e multiplexer unit 4. 

STM-1 RSOH and MSOH. The data X demultiplexed from the received signal Y at 

Theabove-menton^ thc dcmul ^ lcxa U J X m writtCD ^ ^ mem £y 2 as 

position of the so^aUed Jl byte in me figure Using tidbit 33 ^ ^ with ^ spccd rf ^ 

is possible to specify the header position of data^Wn ^ ^ ^ ^ ^^^^^ , me 

mapping data in tr* higher order viniial container VO* it ^ aofthc ^ mc ^ x 

is not set where the header position of thc data will be. r . , w ^ m . 

Therefore, a pointer is used to show the header position. exunplc shown ui FIG. 1 to FIG. 4. since the 63 

As explained above, synchronous multiplexing has been « "^f," 1 ^ &tcd J rdcr . 1Q «y«*n»noas 

perforrnTduptomeMgherord^regionduttom^ m ^ u ! c S™"!??* ^JS? v *° 

Stion of digital communicatioo networks in recent years. valucs «■ *» 15 nwoitored for each tributary unit. 

As a result, asynchronization among transmission appara- The data X written into the memory 2 are read out from 

tuses sometimes causes trouble. Accordingly, it has become memory 2 in synchronization with the master clock (f ) 

necessary to check for the establishment of synchronization « of ** nodc l****™**™ apparatus) from the clock source 

by simple methods or apparatuses. Below, an explanation 3 and are multiplexed into the signal V at the multiplexer 

will be given of the present invention which enables the »™t 4. At this time, the multiplexer unit 4 confirms the 

establishment of synchronization to be checked by a simple position of the headers of the data X. adds pointer values p* 

mct | 10< j showing those positions, and monitors the values. 

FIG. 5A is a view of the configuration of a node » In the example shown in FIG. 1 to FIG. 4. the multiplexer 

(transmission apparatus) in a synchronous communication 4 successively reads out 63 tributary units from the 

network to which the principle of the present invention is memory 2 in the same order as at the demultiplexer unit 1 

applied. FIG. 5B is a view of the signal formats of a received side and once again composes the synchronous transfer 

signal Y and transmission signal V io FIG- 5A. module STM-1. At this time, it adds VI as the pointer values 

The method of the present invention checks the establish- 55 P in mc samc ordcr ' 

ment of synchronization at a node of a synchronous com- The comparing unit 5 compares the pointer values a being 

raunication network which communicates by sending and monitored in the demultiplexer unit 1 and the pointer values 

receiving frames formed by successively adding pointers (ou P being monitored in the multiplexer unit 4 and, if a and p* 

P) to a plurality of data X to show their header positions. It match, it is judged that synchronization has been established 

compares the values a of the pointers added to the data X 60 between the node (transmission apparatus) and the apparatus 

received from an opposing node (not shown in FIG. 5A. but of &e opposing node transmitting the received signal Y. 

at the left side of the figure) side and the values p of the Further, when a and (S do not match, it is judged that there 

pointers to be added to the data X transmitted from the home is a difference in the speed of the docks from the dock 

node (node shown in the figure) and. when detecting that the sources used as master clocks by the transmission appara- 

two values (a, p) match, recognizes that synchronization has 65 ™«s and therefore synchronization is not established, 

been established between the opposing node and the home This processing is performed on the pointer values a and 

node. P with respect to the header positions A lt A 2 of the data 
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X p x 2 in the received signal Y and the header positions At the node 30. the data X are taken out from Die signal 

Bp B ... of the data X,, Xj in the transmission signal V, the added pointers (p) are demultiplexed and removed by 

V and the coincidence of the results of comparison of a and the pointer deleting unit 32, and thus the data X arc 
p* is watched so as to check the establishment of synchro- extracted. 

Dilation (between the opposing node and home node). These s Note that in FIG. 6. the nodes 10. 20. and 30 are similar 

X t . Xj A,. A 3 Bj. Bj, . , . are as shown in FIG. to each other in configuration, but only the portions required 

SB. Note that the pointer shown in FIG. 5B corresponds, for for explaining the operations at the nodes are drawn. As 

example, to the pointer AIM PTR io FIG. 4. The RSOH and mentioned above, the node 20 of FIG. 6 corresponds to the 

MSOH adjoining this AIM PTR are omitted from the node shown in FIG. 5A. and the members 1. 2, 3. 4, and 5 

illustration in FIG. 5B. io in FIG. 5 A correspond to the members 21. 22. 23. 24. and 

In this case, the pointers are compared between the 25 in FIG. 6. 

overheads of the received signal and transmission signal in (his way. according to the present invention, by detect- 

given path overheads or section overheads of the same level ing the coincidence of the pointers of the received signal and 

For example, referring to FIG. 2. a comparison is made by the transmission signal, it is possible to confirm the state of 

the comparing unit 5 between the pointers belonging to the t 5 synchronization between nodes (transmission apparatuses), 

path overhead VC-12 FOH level. This information confirming synchronization can be used as 

FIG. 6 is a view of an embodiment of the present alarm information for synchronization between transmission 

invention. Reference numerals 10. 20. and 30 show nodes apparatuses. For example, it may be used for transmitting a 

(offices) in a synchronous communication network, ft shows message showing that they are out of synchronization, 

only the important parts of the configuration at the nodes 20 FIG. 7 is a view of an example of the configuration of a 

when transmitting data X from the node 10 through the node demultiplexer unit in the present invention. Reference 

20 to the node 30. Further. 40 shows a transmission line numeral 61 is a timing generating unit which generates a 

connecting the node 10 and the node 20. while 50 shows a timing signal from the received signal. 62 is a demultiplex 

transmission line connecting the node 20 and the node 30. function unit which performs a demultiplex function for 

At the opposing node 10, reference numeral 11 is a dock 23 demultiplexing the received signal into the data and 

source which supplies a master clock (f) at the node 10. pointers, and 63 is a holding unit for holding the demulti- 

Reference numeral 12 is a multiplexer unit (MUX) which plexed pointers. 

multiplexes the data X to form a transmission signal Y. At the timing generating unit 61. the timing component is 

Reference numeral 13 is a pointer adding unit which adds ^ extracted from the received signal to generate the timing 

the pointer values a to the data X. signal. At the demultiplex function unit 62. the timing signal 

At the home node 20 (corresponding to FIG. 5A), 21 is a produced by the timing generating unit 61 is used to demul- 

deraultiplexer unit (DMUX) which receives the signal Y tiplex the received signal into the data portion and the 

from the node 20 and demultiplexes the data X and the pointers showing the headers of the data. The data are then 

pointers (a). Reference numeral 23 is a clock source which 35 output and the demultiplexed pointers held temporarily in 

supplies the master clock (f) at the node 20. Reference the holding unit 63. 

numeral 22 is a memory which holds the data X demulti- FIG. 8 is a view of an example of the configuration of a 

plexed at the demultiplexer unit 21. Reference numeral 24 is multiplexer unit in the present invention. In the figure, 

a multiplexer unit (MUX) which remaps the data X read out reference numeral 71 is a pointer generating unit for gen- 

from the memory 2 in synchronization with the clock (f ) and ^ erating pointers to be added to data read out from the 

adds the pointer values p* to the data X to form the trans- memory. 72 is a multiplex function unit which functions to 

mission signal Y\ Reference numeral 25 is a comparing unit multiplex the data and the pointers and produce a transmis- 

which compares the pointer values a and the pointer values sion signal, and 73 is a holding unit for holding the pointers 

p\ produced in the pointer generating unit. 

At the node (downstream side) 30. 31 is a demultiplexer 45 The pointer generating unit 71 produces pointers corre- 

unit (DMUX) which receives the signal Y* and demulti- sponding to the data successively read out from the memory, 

plexes it to the data X and the pointers (fl). Reference At the multiplex function unit 72, the corresponding pointers 

numeral 32 is a pointer deleting unit which removes the are added to the data successively read out from the memory 

demultiplexed pointers and outputs the data X. so as to form frames and produce transmission signals and 

At the opposing node 10, the data X are multiplexed to the so the added pointers are temporarily held in the holding unit 

signal Y (mapped into Y) in synchronization with the master 73. 

clock (f). At this time, the pointer values a showing the FIG. 9 is a view of an example of the application of the 

headers of the data X in the signal Y are added. The signal present invention. This figure illustrates the structure of the 

Y sent from the node 10 is transmitted through the trans- home node 20 shown in FIG. 6. Reference numeral 26 is an 
mission line 40 and received at the home node 20. 55 external clock source provided at the node 20, the signals Z 

At the home node 20, the data X is taken out from the and Z are downstream signals when the signals Y and Y* at 

signal Y and held in the memory 23 and the values a of the the node 20 are made the upstream signals, for example, and 

pointers which have been added are monitored. Further, the P is a signal from a subnode (tributary) merging at the node 

data X are multiplexed by remapping into the signal Y* in 20. Further. "1" is the riming signal extracted from the 

synchronization with the master clock (f ) of the node 20. At 60 received signal Y. "2" is the master dock from the external 

this time, the values a of the pointers and the values P of the clock source 26. "3" is the timing signal extracted from the 

pointers are compared and if a does not equal p\ it is judged signal P from the subnode. and "4" is the timing signal 

that the clock f does not equal f . By this, it is confirmed that extracted from the downstream signal Z> 

the docks of the node 10 and the node 20 are not synchro- Assuming now that the timing signal "1" from the 

nized. The signal Y* sent from the node 20 is sent through the 63 received signal Y is used as the master clock source (f). 

transmission line 50 and received at the downstream node since the pointers a are always equal to p\ there is synchro- 

30. nization with the opposing node sending out the signal Y. 
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However, when the timing signal "4™ is used as the master 
clock source (f). when synchronization is not established in 
the communication artwork as a whole due to some reason 
or another, a frequency difference is created between the 
clock of the signal Y clock source and the master clock (f ) 5 
at the node 20. 

Therefore, a plurality of clock sources are provided at the 
home node. These plurality off clock sources are periodically 
successively selected and used. The one clock source giving 
the result with the least nonco incidence in the comparison is lQ 
finally selected and used as the representative clock source 
for the home node. 

That is, the pointer values in the case of use of the values 
of the plurality of clock sources able to be periodically 
selected by the transmission apparatuses are compared and 
the single clock source giving the smallest frequency dif- 
ference is selected as the master clock (f ) of the apparatus. 
In this way. by making use of the pointer values to check the 
quality of the dock source, it is possible to continuously 
establish a synchronous network. 

Note thai in mis case, if use is made of the timing signal 20 
"1" at all times, it might be thought that do problem would 
arise, but when (here is some problem such as an abnormal- 
ity in the apparatus at the opposing node side of the signal 
Y, it is not possible to use the timing signal "1". By setting 
up the system as in the present example of application, it is 23 
possible to select the more accurate clock source at all times. 

Finally, more specific examples of the configuration of 
several of the constituent elements shown in FIG. 6. FIG. 7. 
and FIG. 8 will be shown. 

FIG. 10 is a view of a more specific example of the 30 
configuration of the node 10 in FIG. 6. In particular, it shows 
more specifically the pointer adding unit 13 of FIG. 6. As 
illustrated, the pointer adding unit 13 is comprised of a 
memory 81 for holding temporarily the data X. a pointer 
calculation unit 82 for calculating (counting) where the 33 
headers of the data X are so as to calculate the pointer values 
p. and a pointer insertion unit 83 for writing the values p* in 
pointer regions. After this, it is connected through an over- 
head processing unit 84 to a multiplexer unit IX 

The format of the signal Y is formed in synchronization 
with the master clock (0 and is sent to the node 20, but in 
this case a fixed value is always used as the values a of the 
pointers to be Inserted. The reason is that the data X from the 
memory 81 is also synchronized with the clock (f)- 

FIG. 11 is a view of a more specific example of the 
configuration of the node 30 in FIG. 6. In particular, it shows 
more specifically the pointer deleting unit 32 in FIG. 6. 

The signal output from the demultiplexer unit 31 passes 
through the overhead processing unit 87. then the pointer ^ 
detection unit 85 detects its pointer portions. Further, the 
values p* written into the pointers are read out. 

On the other hand, the above-mentioned signal is stored 
temporarily in the memory 86 as well. Only the data X to be 
dropped at the node 30 are read out from the memory 86 by 55 
accessing by the pointer values p\ The portions accessed by 
P arc the header positions of the data X. 

FIG. 12 is a view of a more specific example of the node 
20 in FIG. 6. Note that constituent elements substantially the 
same as those already explained arc given the same refer- 60 
ence numerals. 

The demultiplexer unit 21 of FIG. 6 is comprised of a 
demultiplex function unit 62 (FIG. 7). a tuning generating 
unit 61 (FIG. 7). a pointer detection unit 85 (FIG. 11). and 
a holding unit 63 (FIG. 7). Note that in this figure, the 65 
detection unit 85 is drawn pulled out from the function unit 
62. 


40 


45 


The multiplexer unit 24 of FIG. 6 is comprised of a 
pointer calculation unit 82 (FIG. 10) and pointer insertion 
unit 83 (FIG. It) forming the pointer generating unit 71 
shown in FIG. 8. a multiplex function unit 72 (FIG. 8), and 
a holding unit 73 (FIG. 8). 

In a specific example of the present invention, provision 
is made of a header position memory 91 between the pointer 
detection unit 85 and pointer calculation unit 82. Each time 
the pointer detection unit 85 detects a pointer, the detection 
is written, in a form of pulse, into the memory 91 in a time 
series. The timing at this time is the clock (f). 

On the other hand, the detection pulse stored in the 
memory 91 is read out in time series in synchronization with 
the clock (f ). In synchronization with this readout operation, 
the header positions of the data read out from the memory 
22 are calculated (counted) to find the values p" of the 
pointers. 

FIG. 13 is a view of an example of a demultiplex function 
unit shown in FIG. 7 and FIG. 12. Constituent elements 
substantially the same as those already explained are given 
the same reference numerals. In the demultiplex function 
unit 62. the data reproducing unit 92 and the frame syn- 
chronization unit 93 for detecting the frame byte (header) of 
the received data are constituent elements shown for the first 
time in this figure. Note that the pointer detection unit 85 
operates to detect pointer areas (PTR) located at the portions 
separated by a certain predetermined number of bytes from 
the frame byte (header) detected by the unit 93. 

As explained above, the present invention enables con- 
tinuous confirmation of the synchronization among trans- 
mission apparatuses in a synchronous communication net- 
work and therefore contributes greatly to the improvement 
of the reliability of communication networks overall. 

We claim: 

1. A method for checking the establishment of synchro- 
nization at a node in a synchronous communication network 
which performs communication by sending and receiving 
multiplexed frames, comprising the steps of: 
forming said frames by successively adding pointers 
showing header positions to a plurality of data portions; 
comparing at said node between the value of a pointer 
included in each data portion received from an oppos- 
ing node side and the value of a pointer to be added to 
each data portion, which is the same as the received 
data portion, and to be sent from a home node; 
detecting whether the compared values coincide, synchro- 
nization being established between the opposing node 
and the home node when said compared values coin- 
cide; 

demultiplexing each of said frames received at said home 
node in a received signal from the opposing node into 
said data portions and said pointers in correspondence 
with a timing component in the received frames; 

holding the data in a memory device. 

successively reading out the held data portions in corre- 
spondence with a clock of a dock source in the home 
node, adding pointers to the read out data portions at 
the home node showing headers of the data portions to 
be sent and multiplexing to form frames of a transmis- 
sion signal from said home node. 

comparing the pointers demultiplexed from the received 
frames and the pointers added to the transmission 
signal, and 

detecting that a clock of the opposing node that provided 
the received signal and the clock of the home node are 
synchronized when the compared pointers are found to 
coincide. 
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wherein a plurality of dock sources arc provided at the 
home node and 

a further step of periodically successively selecting and 
using the plurality of dock sources, one of said plu- 
rality of dock sources giving a result with least dod- 3 
coincidence in the comparison being finally selected 
and used as a representative deck source for the home 
node. 

2. Ad apparatus for checking for the establishment of 
synchronization at a node in a synchronous communication 10 
network which performs communication by sending signals 
and receiving signals of multiplexed frames formed by 
sticcessivdy adding pointers showing header positions to a 
plurality of data portions, comprising: 

a demultiplexer uai t ( 1) al a bome node for derauttiptex- 13 
ing a received signal sent from a transmission node into 
data portions and pointers in correspondence with a 
liming of the received signal, 
a memory (2) for holding the demultiplexed data portions. ^ 
a multiplexer unit (4) for successively reading out the held 
data from said memory in time with a dock of a dock 
source (3) of the home node, for adding pointers 
showing headers of the read out data and for multi- 
plexing said read-out data and added pointers to pre- 25 
pare a transmission signal to be sent from said home 
node, and 

a comparing unit (5) for comparing the pointers separated 
from said received signal by the demultiplexer unit (1) 
and the pointers added to said transmission signal by 30 
the multiplexer unit (4). so as to find whether or not the 
synchronization between a dock of the transmission 
node of the received signal and the dock of the home 
node is established by the resultant comparison and. if 
it is not established, said clock of said home node is 35 
adjusted without adjusting the pointers. 

3. An apparatus for checking for the establishment of 
synchronization at a node in a synchronous communication 
network which performs communication by sending signals 
and receiving signals of multiplexed frames formed by 40 
successively adding pointers showing header positions to a 
plurality of data portions, comprising: 

a demultiplexer unit (1) at a home node for demultiplex- 
ing a received signal sent from a transmission node into 
data portions and pointers in correspondence with a 45 
timing component in the received signal. 

a memory (2) for holding the demultiplexed data portions. 

a multiplexer unit (4) for success! vdy reading out the held 
data from said memory in time with a dock of a dock x 
source (3) of the home node, for adding pointers 
showing headers of the read out data and for multi- 
plexing said read-out data and added pointers to pre- 
pare a transmission signal to be sent from said home 
node, and 


a comparing unit (5) for comparing the pointers separated 
from said received signal by the demultiplexer unit (1) 
and the pointers added to said transmission signal by 
the multiplexer unit (4), 

synchronization between a clock of the transmission node 
of the received signal and the clock of the home node 
being detected by coincidence in the comparison. 

wherein said demultiplexer unit (1) includes a demulti- 
plex function unit for said demultiplexing of the 
received signal into said data portions and pointers, and 
a first holding unit (63) for holding the demultiplexed 
pointers. 

said multiplexer unit (4) includes a multiplex function 
unit (72) for successively reading out the data portions 
held in the memory (2). timed by the dock of the home 
node, and for successively adding said pointers to said 
read-out data portions indicating the respective headers 
of the read-out data portions and multiplexing said 
read-out data portions and pointers to prepare said 
transmission signal, and indudes a second holding unit 
for holding the added pointers, and 

said comparing unit (5) detecting the coincidence of the 
pointers held in the first holding unit (63) and the 
pointers held in the second holding unit (73) coinci- 
dence of said pointers corresponding to synchroniza- 
tion between the dock of the transmission node of the 
received signal and the dock of the home node. 

4. An apparatus for checking for the establishment of 
synchronization as set forth in claim 3. wherein said mul- 
tiplexer unit (4) indudes a pointer generating unit (71) for 
adding pointers to said read-out data portions. 

5. Ad apparatus for checking far the establishment of 
synchronization as set forth in claim 4. further comprising: 

a pointer detection unit (85) provided in said demulti- 
plexer unit for detecting said received pointers in said 
received signal. 

a pointer calculation unit (82) provided in said pointer 
generating unit (71) for adding said added pointers to 
said data portions read-out from memory, and 

a header position memory, connected between said 
pointer detection unit and said pointer calculation unit 
for writing detection pulses by a first clock each time 
the pointer detection unit detects a pointer, reading out 
detection pulses by a second clock, and causing the 
calculation of the header position of the received data 
at the pointer calculation unit 

6. An apparatus for checking for the establishment of 
synchronization as set forth in claim 3. wherein said demul- 
tiplexer unit (1) includes a timing generating unit (61) for 
extracting the timing component of said received signal. 
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[57] ABSTRACT 

Apparatus and methods for distributing synchronization 
throughout a network is disclosed. The distribution of the 
synchronization is through the use of generating a reference 
timing signal, and by counting the line clock pulses between 
the start of a frame and the timing reference signal pulse at 
a first office and that count is then encoded and transmitted 
to the next office. Al the next office, the transmitted count is 
decoded and used for regenerating synchronization by 
counting a number of received line clock pulses from the 
start of the frame to regenerate the reference timing signal. 
Particular criteria for selecting the frequencies for the timing 
reference signal are disclosed. 
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DISTRIBUTION OF SYNCHRONIZATION IN 
A SYNCHRONOUS OPTICAL 
ENVIRONMENT 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention relates to providing synchronization dis- 
tribution throughout a network and in particular relates to 
providing this distribution in a synchronous optical commu- 
nications environment prevalent in telephone networks. 

2. Description of the Environment 

In digital telephone networks, the network is comprised of 
hundreds or even thousands of offices or nodes such as 
shown in simplified form in FIG. 1. The network 10 has a 
plurality of offices 12, 14, 16, 18, 20. Each node has a local 
timing source 12a, 14a, 16a, 18a, 20a, commonly called a 
BITS for Building late grated Timing Supply. Also, each 
node has a variety of equipment such as switches, optical 
multiplexers, channel blanks, etc. commonly referred to as 
network elements (NE's) 12f>, 14£>, 16b, ISb, 20b, with the 
timing for each network element within the office being 
supplied by the office's BITS. The various offices within the 
network are connected by copper or optical fiber links 22 
called facilities. Unlike the earlier versions of the copper 
based networks where the facilities formed a mesh type 
network with each office being linked to multiple offices by 
several facilities, digital optical networks are arranged in 
chains or rings with facilities tying each office typically to 
the two adjacent offices. 

Further, in a typical digital network, there are a plurality 
of primary reference source clocks called PRS clocks. 
Typically, the PRS clocks are implemented using cesium 
beam or GPS receiver technology. The PRS clocks serve as 
master clocks and provide a timing reference for the remain- 
der of the network. The PRS timing is communicated over 
the facilities to different nodes to permit synchronization 
between various nodes within the network. 

The earlier (non-standard) versions of the optical fiber 
network employed asynchronous bit stuffing techniques to 
multiplex the input tributary signals onto the optical line. 
The distribution of the timing reference in such a network 
may be realized using an embedded DS1 signal, as shown in 
FIG. 2. The PRS timing 30 in an office 32 is provided to the 
BITS 35 and then to fiber multiplexer 36 in a first office and 
communicated to the next office 40 over the embedded DS1 
signal in the optical facility 38. Further, the fiber multiplexer 
42 at the next office 40 recovers the DS1 clock 44 and passes 
that recovered clock to the BITS 46 of the second office and 
to the fiber multiplexer 48 for transmission over the next 
facility in the chain to the next office. Since the BITS clock 
46 is not used for generating the line timing signal SO 
provided to the next office in the chain (not shown), inac- 
curacies in the timing reference communicated to the BITS 
■ timing in intervening offices do not effect the timing refer- 
ence communicated to the BITS of the successive offices. 
Thus, if the BITS timing reference in the second office 
malfunctions, the synchronization of the successive nodes or 
offices (not shown) in the network is unaffected. Therefore, 
each of the nodes or offices in the network may be thought 
of receiving its synchronization liming directly from the 
offices containing the PRS. Where each of the nodes of the 
network is receiving the timing reference directly from the 
PRS, synchronization may be thought of as being a) the 
same level. Such distribution schemes of synchronization 
are referred to as being fiat. 

Although the method described above yields the desirable 
flat synchronization distribution system, it is not deployed 
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extensively in the telephone network for two reasons. First, 
the bit stuffing operation performed at each node adds jitter 
to the embedded DS1 synchronization reference. This may 
render the DS1 signal unusable ( as a timing reference after 

5 it traverses a few nodes. Second, and more important, the 
nonstandard asynchronous optical fiber systems are being 
replaced by the recently developed standard synchronous 
optical network technologies, designated as SONET or 
SDH. The method of distributing the synchronization ref- 

io erence using an embedded DS1 signal does not work prop- 
erly in the SONET environment, as explained below. 

In a SONET multiplexer, the output optical line clock is 
normally synchronized to the office BITS clock. The rate 
variations between the input tributaries and the output line 

15 signal are accommodated by a byte stuffing process known 
as pointer adjustment. The eight bit phase movements 
caused by the pointer adjustments can be large enough to 
render the embedded DS1 timing reference incapable of 
adequately transporting the synchronization information. 

20 Hence, the standards organizations (ANSI and the ITU) 
recommend that a DS1 signal embedded within a SONET 
line signal not be used for synchronization distribution. 
Instead they recommend the use of the recovered optical line 
clock to generate a derived DS1 synchronization signal. This 

25 derived DS1 signal serves as the synchronization reference 
input to the office BITS clock. 

The use of the derived DS1 to distribute synchronization 
references, however, implies a hierarchical synchronization 
network. In such a network, the BITS clock at an interme- 

30 diate node is not synchronized directly to the PRS but is 
instead synchronized to the liming reference supplied by the 
BITS clock in the previous node. This hierarchical scheme 
for the distribution of synchronization signals has marry 
shortcomings. 

First, administrative controls are required to ensure that a 
higher quality BITS clock (lower stratum number) does not 
accept timing from a lower quality BITS clock. Second, the 
cascade of clocks created by the hierarchical chain can 

^ impair the timing reference traversing the network. Third, if 
a BITS clock fails anywhere in the chain, all the downstream 
clocks will lose synchronization. And finally, this scheme is 
prone to the inadvertent creation of timing loops, especially 
under facility failure conditions. (A timing loop occurs when 

45 timing from a first node is passed to the second node and 
then the timing is fed back through a chain of one or more 
additional nodes to Ihc first node so that the first node is 
synchronizing its timing to itself. Such a situation is clearly 
undesirable since all the nodes involved in the timing loop 

50 will be isolated from the PRS). 

Synchronization messaging is a solution recommended by 
the standards organizations to alleviate some of the short- 
comings delineated above. In this method, the status of the 
clock that generates the timing reference at a particular node 

55 is communicated to the clocks and network elements at other 
nodes over a messaging channel. The clocks at these other 
nodes will then decide, in an intelligent manner, whether 
they should synchronize to one of the incoming liming 
references, or whether they should operate in a holdover 

60 mode. However, the synchronization messaging scheme 
docs not cure all the problems created by the hierarchical 
synchronization distribution network. Furthermore, imple- 
mentation of this messaging scheme will be expensive as it 
involves the retrofitting of the existing BITS clocks and the 

6 5 SONET network elements to provide this capability. 

Therefore, it is the first objective of this invention to 
provide a method for transporting network synchronization 
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reference signals over the existing SONET network using a an embodiment of the invention, as applied in a SONET or 

flat distribution scheme. It is a second objective of this similar synchronous optica] environment. A portion 100 of 

invention to distribute these synchronization reference sig- the network is shown in FIG. 3. Each node or office 102, 

nals without incurring the problems associated with the 104, in the network has a BITS clock source 106, 108 and 

hierarchical scheme. It is yet a third objective of this 5 at least one node has a PRS 110 directly controlling the BITS 

invention to achieve the flat synchronization distribution timing 106 at the same office 102. Each node receives and 

system without requiring substantial hardware investment or transmits a line clock LCLK 114 over a facility such as an 

retrofitting costs. optical fiber 112 linking nodes. Each node also receives and 

SUMMARY OF THE INVENTION transmits frames at a nominal rate of eight thousand times a 

1 n second with the frame containing control information and 

These and other objects are achieved by relying on two 10 data according to me established network protocol. In the 

u mi 2&2S e ? aVaUabk V ach 0&X '' e ^ [int dOC , k aDd SONET environment, the duration of a frame is nominally 

the SONET frame ttming. A timing reference signal syn- one huQdred ^ fiye microseconds . A locaUy generated 

chromzed to the PRS and at a frequency that is at least ^ refcrcnce ^ j M6 ^ frQm me mTS ^ 

slightly less than the frame rate is generated at the original- . t: ... . . WII 5 

ing PRS site. The line clock is then used to determine the is <* e *™ "P" 1 ? ^.Jf T ¥ * ^ M " Z 

infervalbetween the start of the frame and an edge of the ^ SONET) is also provided in each node having a 

timing reference signal. This timing difference is encoded P * S ' ^ sen f "1™^ b V me ° ffice or Qode 1Q2 * 

and transmitted in an overhead channel and may be decoded * signal LFRM 122. The frame start signd and 

at the next node. The next node may then recover the timing ^ cIoc \ can * obtained from the add-drop multiplexer 

reference for use in its own BITS timing and for transmis- 20 < ADM ' not shown ) at ^ office - 

sion on to the next node. An encoder 120 measures the difference in timing 

Therefore, a flat synchronization structure is created as betw f n ! he f f ° f 5^° f ! ndi ? ated by ?* frame star [ 

each node in the network depends for its timing upon the J^W* 1 . ?' l ^l? 1 * n "**&f 

original PRS instead of the intervening nodes. Further, this ™* Umm S difference may be obtained by counting the 

flat structure eliminates any possibility of liming loops. 25 ^J^J^ between the start of the frame as indicated 

i> cwt.- „». ■ • . _i . - ^ . by LFRM 122 and a pulse edge of the locally generated 

To fit this approach in existing network structures without ,. . , f .. . ,. a J & . , 

, , . . , v * , y . „. „ , timing reference signal. The liming difference represented 

substantial hardware costs, a few counters, flip-flops, and . 4 , . , .° j j • „ j c 

. . , * 11 *.c .- ■ • 1 -r by this count may be encoded mto reserved control bytes of 

gates can be used to Generate all of the timing signals. To * , AAS , 4 . . . .„ . 

6 . . ... j j - .Pa- the message frame 115 and may then be transmitted over the 

reduce messaging overhead, the encoded tuning difference f .... * , , *. \ nA . 0-^™ . - 

b ., , . . , , j in facihty 112 to the next office 104 in the SONET chain or 

can be transmitted over multiple frames in currently used 30 . ... . «- j . t 

. , .... onvirT . ♦ ' nng. At this second office, a decoder 126 uses the transmit - 

control bytes reserved in the SONET architecture. t . . t , . ' , t . t , f «- 

J ted line clock 114 that has been transmitted from the office 

DESCRIPTION OF THE DRAWINGS 102 and recovered in the office 104 by the ADM (not shown) 

FIG. 1 is a diagram of a simple prior art telephone alon g with a hne frame staTt si g naJ 122 ' that nas been 

network 35 reconstructed by the office ADM according to well known 

FIG. 2 is a diagram of Ihe synchronization distribution techniques in the art. In a manner explained below, the local 

scheme in a prior art asynchronous network. Umrng reference signal U6 can be regenerated for example, 

FIG. 3 is a block diagram for an encoder and a decoder for ^ flying the period of the line clock LCLK 114 by the 

... . , . ,. transmitted count 115 to generate a pulse in a manner that 

an embodiment of the invention. ... , , .. . . & , , .. . . r 

* > , JT , . . , . . n will be desenbed in more detail below. 

FIGS. 4A and 4B arc timing diagrams relating to the * u _ . , , . . . . . . ... 

... , - fat»* b That regenerated liming signal 116' may then be supplied 

embodiment of HG. 3. . * i-im .u T 1 • .u * . c .u 

,. t ., OJ . P . LJ . e to a further encoder 120' that also receives the start of the 

FIG. 5 is a functional flow diagram for the embodiment of frame signal m g Cnerated by me office for ^ framcs to bc 

„..„.. transmitted. The line clock 114" for transmission to the next 

FIGS. 6A and 6b are schematics of circuits for retiming ^ nodfi fa (he chain of ^ nelWQrk (nol shown) u also 

the timing reference signal at the encoder. provided to the encoder 120' from the ADM (not shown). 

FIG. 7A is a schematic of a circuit in the encoder for difference between the start of the frame pulse LFRM 

measuring the timing difference between the start of the 124 and thc rc g Cncra tcd timing signal 116* may bc counted 

frame and the timing reference signal. ^th the line clock to provide a further count 115' for 

FIG. 7B is the timing diagram for FIG. 7A 50 transmission over thc facility (not shown). 

FIG. 8 is a schematic of a circuit in the encoder for FIG. 4A shows a liming chart relevant to time measure- 
generating a flag to indicate in which frame an edge of thc ment at the first office 102. The 51.84 MHz hne clock LCLK 
timing reference signal has occurred. 114 provides the fundamental reference for counting periods 

FIG. 9A shows a circuit in the encoder for sampling the or bit times for the time measurement. The start of each 

measured timing difference and the flag. 55 successive framcs N-l, N, N+l, N+2 is indicated by the 

FIG. 9B is the timing diagram for FIG. 9A. rising edge of a pulse in the frame start signal LFRM 122. 

FIG. 10A is a circuit in the decoder for generating the Since a frame has a duration of 125 microseconds, there are 

timing reference signal in the decoder. 64S0 possible periods of the line clock LCLK in which an 

FIG. 10B is the timing diagram for FIG. 10A. ed & e of At ti^g reference signal 116 can occur. In ihe 

FIG. 11 is a circuit for generating a substitute for the «> instance in frame N, the edge occurs during the fourth bit 

timing reference signaJ when certain error conditions are Um r c measured in units of the 51.84 MHz clock so a count 

detected °^ ™ ur wou ^ °e encoded. During the next frame (N+l) or 

some subsequent frame, that count may be transmitted to the 

DETAILED DESCRIPTION OF THE ncxt nodc 104 over the link 112. 

INVENTION 6S Upon rece ip t 0 f me count, the next node 104 in the 

The embodiments of the invention involve transmission network will generate a rising edge of a reconstructed timing 

of synchronization of timing. FIG. 3 is useful in explaining reference signal 116' at the start of the fourth received bit 
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lime in the N+K frame as shown in FIG. 4B. In particular, 
the decoder 126 will receive the regenerated line clock 
LCLK 114' generated by the ADM at the office 104 and 
count a number of cycles of that clock equal to the received 
count. At that point, the decoder will generate an edge in a 
regenerated timing reference signal 116' that may be used in 
the office 104 for synchronizing the BITS 108 to the PRS 
102. In addition, the line clock, the regenerated timing signal 
116' and the local frame start signal 124 from the second 
office's 104 SONET ADM (not shown) may be used for 
measuring the difference and transmitting a count to the next 
office so that it may also generate a local version of the 
timing signal. Since each version of the timing signal is only 
dependent upon the PRS timing and not the BITS timing of 
each local office in the chain, distribution of the timing tied 
to the timing reference is flat and avoids both timing loops 
and hierarchy problems. 

FIG. 5 shows a functional Sow diagram of distribution of 
timing throughout the network. Successive offices N-l t N 
and N+1 in a network perform the function of measuring the 
timing difference between a timing reference signal and the 
line clock and the start of the frame using the transmit line 
clock. The measured time difference at node N-l is encoded 
and then transmitted over a link 150. At the next node, or 
office N, the line clock and the frame timing are recovered, 
the transmitted count is decoded and used to regenerate the 
timing reference signal. This regenerated timing reference 
signal is supplied to the BITS at office N for synchroniza- 
tion. This regenerated timing reference signal at office N is 
also supplied for measuring the timing difference with the 
start of the frame at office N to be transmitted to the next 
node using the transmit line clock and frame timing of office 
N. The timing difference measured at office N is encoded 
and transmitted over the facility for subsequent recovery, 
decoding and regeneration at office N+l in a like manner. 

Selection of the appropriate criteria for the local timing 
reference signal is based upon several factors. First, the local 
timing signal pulse edge should occur no more than once 
each frame. Therefore, the local timing signal should have a 
rate of less than or equal to the frequency of a frame; e.g., 40 
8 KHz in a SONET network. However, since the frame rate 
may be at a slightly lower rate and still be within the SONET 
specification, the local timing reference signal should pref- 
erably occur at a rate that is less than the minimum permitted 
frequency of the frames. Further, the frame rate and the 
timing signal rale should not be harmonics of each other. 
Optimally, they should be "as prime as possible" with 
respect to the measurement rate of the line clock, which is 
51.84 MHz. In particular, the highest common factor of the 
frame rate of 8 KHz and the timing reference signal should 
be as low as possible to prevent the development of beats 
occurring with such sampling with the line clock. In 
addition, the timing rate signal should preferably be at a 
frequency readily obtainable from frequencies in the office 
such as the BITS signal. For this reason, a timing rate of 7.72 
KHz or an integral submultiple thereof (i.e., 3.86 KHz, 1.93 
KHz, 0.965 KHz) are among the preferred frequencies for a 
SONET network. These frequencies can be readily gener- 
ated from the 1.544 MHz BITS timing sign a] available in 
each office since 7.72 KHz is obtained readily by dividing 
the BITS signal by two hundred. In fact, for reasons dis- 
cussed later, 1.93 KHz may preferably be used for optimal 
encoding in an overhead byte for transmission over a 
SONET network. 

A further advantage of the 7.72 KHz timing signal or 
some integral submultiple thereof is avoidance of metastable 
states resulting from the synchronous nature of the various 
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signals and the switching speed of the digital logic involved. 
In particular, with the line clock of 51.84 MHz, there is a 
window around each edge of the line clock in which the 
occurrence of an edge of the timing signal may not be 
detected due to the transistor switching delays inherent in 
the digital logic. Such metastable conditions would result in 
a delay in detecting the edge and hence inject a one clock 
period pulse offset in detecting the count representing the 
occurrence of the timing edge. If a signal having the same 
frequency as the frame rate, or harmonic thereof, is selected, 
this metastable condition can persist over a substantial 
period of lime. With the selection of 7.72 KHz or an integer 
submultiple of that frequency, any metastable events will be 
one time events that can readily be eliminated at the next 
office through the use of a phase lock loop in the generation 
of the timing reference signal. By use of 7.72 KHz or integer 
submultiples of 7.72 KHz, the edge to edge change in the 
timing reference signal ensures that if a timing signal edge 
occurs in the metastable region of the edge in the line clock 
for the logic, the next occurrence of the timing reference 
signal edge will not be during the metastable region. 

FIGS. 6A through 11 show various circuitry and associ- 
ated timing diagrams for generating and regenerating the 
timing reference signal at the various nodes throughout the 
network. It is assumed that in each of these circuits, the 
circuit components are synchronous. First, the timing signal 
REF should preferably be retimed to the timing of the line 
clock LCLK. FIG. 6 A shows a circuit for generating such 
retiming while reducing the likelihood of a metastable 
condition. The circuit comprises three edge triggered D 
flip-flops and the retimed timing signal output is REFRT and 
its complement, REFRT__L. FIG. 6B shows an alternative 
version of such a circuit. 

FIG. 7A shows a circuit for generating the count repre- 
senting the timing difference in units of the line clock LCLK 
period between the pulse indicating the start of the frame N 
in signal LFRM, and the retimed timing reference signal. 
The rising edge of the start of the frame signal LFRM resets 
a thirteen bit counter 160 that counts the line clock LCLK. 
When (he falling edge of the complementary retimed timing 
signal pulse RFRT__L occurs, it enables the input of a 
thirteen bit register 162 that is coupled to the output BIT 
TIME COUNT of the synch counter 160 and the line clock 
LCLK. The current value from the counter is clocked at this 
point into the shift register. The contents of this register, 
labelled COARSE OFFSET, represent the timing delay 
between the LFRM frame pulse and the retimed timing 
signal in units of time defined by the LCLK period. The 
contents of the counter are held in the register until the next 
edge of the retimed timing signal FIG. 7B depicts the 
associated liming diagram. 

Since the retimed timing reference signal may not have an 
edge in each frame, a flag indicating when an edge has 
occurred is needed. FIG. 8 shows a circuit that is useful for 
generating the flag signal to indicate that an edge in 
RFRT_L occurred during the current frame. The circuit 170 
receives the line clock LCLK, the retimed timing reference 
signal REFRT, the start of the frame signal LFRM, and a 
start of the frame signal delayed by one line clock period 
LFRMD1. Optionally, two inverters 172a and 1726 may be 
provided between D flip flops 174 and 176 that generate a 
flag signal FLAG indicating an edge of the timing reference 
signal has occurred with a true value indicating a flag has 
occurred. 

For subsequent transmission to the next node, the stored 
count must be sampled for encoding and transmitting. The 
flag and the coarse offset value are then stored as a fourteen 
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bit word as shown in FIG. 9 A. The frame start timing signal 
LFRM is delayed by the line clock LCLK in a flip flop 182 
to provide the LFRMD1 signal and that LFRMD1 signal 
enables a D flip-flop 184 and a register 186 circuit that 
receive as their inputs the FLAG and COARSE_OFFSET 
values. These two values are gated with the line clock LCLK 
for processing during the subsequent frame to provide the 
sampled fourteen bit entity FLAG_OFFSET. The FLAG_ 
OFFSET comprises SAMPLED_FLAG and SAMPLED^ 
COARSE_OFFSET. As shown in the timing diagram FIG. 
9B, the FLAG_OFFSET value lags one frame behind where 
the edge of the retimed timing reference signal RFRT occurs 
(assuming an edge occurred during the prior frame). 

If the resultant sampled coarse offset value has the 
sampled flag high, that count value may then be processed 
by the office network element, such as an ADM, for trans- 
mission over the network to another node on the network 
according to the network protocol. For example, using the 
current SONET protocol, the network has a overhead byte 
called the Fl byte that is unused and reserved for future 
applications. Therefore, it is possible to use the Fl byte for 
transmitting the synchronization information. 

Given that the synchronization count (COARSE_ 
OFFSET) uses the SONET standard of an 8 KHz frame rate 
and a line clock of 51.84 MHz, the coarse offset requires 
thirteen bits to transmit the maximum possible count of 
6479. Therefore, at a minimum, two Fl bytes in two separate 
frames may be used for transmitting the information. 

However, for coding accuracy, it is more desirable to 
transmit the information over four frames and therefore use 
four Fl bytes to permit error detection. Therefore, to match 
this transmission rate of four frames, the frequency of the 
timing reference signal should be 1.93 KHz or some integral 
submultiple of that rate. The timing edge that occurs in 
Frame N, is actually encoded and transmitted in a four frame 
sequence over the next four frames, N+l, N+2, N+3, and 
N+4. Regeneration of the edge at the receiving node when 
using this algorithm will occur at a minimum of five frames 
after the edge occurred at the transmitting node. 

A possible format for the Fl bytes is shown in table I 
below: 


Byte 
No. 

MSB 

Bit 6 

Bit5 

Bil 4 Bit 3 Bit 2 

Bit 1 

LSB 

0 

0 

Edge, 

Edge, 

Coarse Oftset 5 LSB 



1 

0 

Edge, 

Edgo, 

Coarse Offset Next 5 Bits 



2 

0 

Edge, 

Edge, 

Coarse Oflsct 3 MSB 

X 

X 

3 

1 

X 

X 

CRC CRC CRC 

CRC 

CRC 


According to this format, the most significant bit in each 
of the first three Fl bytes is set to a logical zero and the most 
significant bit for the last byte is set to 1 so that the last byte 
of a four frame sequence can be readily detected. 
Alternatively, the MSB of the first byte may be set as one to 
mark the start of a sequence and the three remaining bytes 
may be set to zero. Further, the fifth and sixth bits in the first 
three Fl bytes of a four frame sequence are called edge data 
and are used for determining during which frame of the prior 
four frame sequence an edge of the retimed timing reference 
signal occurred. 

In this implementation, the receiving node office com- 
pares the edge data in each of the first three Fl bytes of a 
four frame sequence. Either a majority rules or a require- 
ment of all three edge bit patterns being the same may be 
used to determine in which frame the edge occurred. Trans- 
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milting different values for each of Edge A and Edge, for each 
of the first three frames for the four frame sequence can be 
used to indicate that no edge occurs during the four frame 
sequence, a link has been broken so synchronization to the 
PRS has been lost or a phase slippage greater than the 
predicted amount has occurred. In addition, various such 
error conditions can be passed between the nodes by encod- 
ing such error conditions in the bits labelled with X. 

The value of the coarse offset is encoded into the first 
three Fl bytes of the four frame sequence. Also, a cyclic 
redundancy check (CRC) sequence, or some other error 
detection mechanism, is transmitted in the Fl byte of the 
fourth frame for detecting transmission errors in the trans- 
mitted Coarse Offset. 

The bits labelled with X may be used for a variety of 
optional functions. For example, such bits may be used with 
the bits indicated as CRC for transmitting a nine bit error 
correction code instead of a CRC. A predetermined bit 
pattern for these bits may also indicate a phase slippage at 
the transmitting node that is greater than a predetermined 
threshold or the like to prevent propagation of phase errors. 
Detection of such errors is readily possible by either the 
transmitting or receiving office. In response to such 
conditions, the receiving office may disregard the transmit- 
ted count for a four byte sequence. 

For example, the receiving office may compute an 
expected range of values for the received coarse offset based 
upon the relative frequency of the frame and the timing 
reference signal. The receiving office may disregard the 
received coarse offset if the value is beyond the expected 
range. 

At the receiving node where the timing difference infor- 
mation (the encoded coarse offset count) is received, the 
timing difference information can be used to regenerate a 
timing reference signal with the use of the circuit 200 shown 
in FIG. 10A. In particular, the regenerated received and 
recovered line clock RLCLK and the regenerated received 
frame signal RLFRM are provided from the receiving office 
ADM (not shown). The transmitted Fl bytes are decoded by 
the receiving office in the chain to provide a reconstructed 
version of FLAG_OFFSET. This regenerated flag signal 
can be based upon the edge bits in each of the first three Fl 
bytes in the transmitted four frame sequence. Either a 
majority rules protocol or a requirement of all three sets of 
edge bit patterns matching can be used for determining in 
which frame the edge occurred. To match the flag signal at 
the equality detector, a logical "1" is also provided so that 
the pulse will be generated during the appropriate frame. 
The coarse offset can be obtained by concatenating the three 
portions of the coarse offset transmitted in the four-frame 
sequence. 

The decoded FLAG_OFFSET is clocked into a register 
202 that is enabled by the reconstructed start frame signal. 
Simultaneously, a thirteen bit counter 204 is counting the 
regenerated line clock RLCLK. Both the output of the 
counter 200 and the register 202 arc provided to an equality 
detector 206. The equality detector can then provide a pulse 
when the count contents and the "one" to match the flag and 
the contents of the register are equal. 

The pulse from the output of the equality detector can then 
be provided to a D flip flop 208 to provide the regenerated 
timing reference signal REFREGEN. This regenerated tim- 
ing signal is provided at least two frames plus one regen- 
erated line clock period after the timing edge occurred. If the 
format described above using four frames for transmission 
of the timing difference is used, the delay will be at least five 
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frames plus the one regenerated line clock period. Any other 
delays inherent in communications between the two offices 
may increase that delay. Nonetheless, such a regenerated 
reference timing signal does permit transmission of synchro- 
nization throughout a network in the manner described for 5 
FIGS. 3-5 above. 

Further, although the use of the line clock at 51.84 MHZ 
provides a granularity of about twenty nanoseconds (the 
period of the line clock) with which to measure 
synchronization, this granularity can be reduced. To reduce 10 
the granularity, one may provide the recovered timing ref- 
erence signal REFREGEN to a digital phase lock loop with 
a very narrow bandwidth, for example about one hertz. The 
granularity of the sampling of the frequency with the line 
clock results in a poisson like distribution of error in the 1S 
sampling of the phase relationship between the timing signal 
and the start of the frame pulse. A narrow bandwidth filter 
over the long term filters out virtually all of the phase error 
due to this poisson like distribution arising from the granu- 
larity assuming the PRS signal is highly stable; i.e., main- 20 
taining an accuracy in one part of 10 13 over the course of a 
day. Thus, the use of such narrow bandwidth phase lock 
loops results in a much more tightly controlled synchroni- 
zation once the phase lock loop has stabilized. 

In particular, with a line clock at about a period of twenty 25 
nanoseconds, the best phase synchronization that could be 
attained would be on the order of twenty nanoseconds. 
However, by employing a one hertz bandwidth phase lock 
loop the phase error over the long term can be reduced to 
about one percent of this granularity on or about the order of 30 
0.2 nanoseconds. This phase lock loop can also be used for 
generating the 1.544 MHz timing reference signal required 
by the BITS clock at the node using a standard frequency 
multiplier configuration. 

Further, to achieve phase lock during start up or after 
various transient conditions, it is preferable that the band- 
width of the filter be adaptive as is readily possible with 
digital filters. During start up or after various transient 
conditions, the bandwidth of the loop is opened up, permit- ^ 
ling faster acquisition of phase lock. 

Upon detection of errors in either the CRC or upon 
detection of the predetermined bit pattern in any of the bits 
indicated with an X in Table 1 above indicating an error 
condition, the receiving node can go into a holdover node. 45 
Also, for such holdover conditions, the regenerated value 
can be ignored by the phase lock loop and the system can 
resort to the predicted values that may be generated by 
knowing the frequency of the frame rate and the timing 
reference signaL Alternatively, with such phase lock loops, 50 
the loop can be held at a nominal frequency until the cause 
of the holdover condition is alleviated. 

Also, the BITS clock can also be used for providing the 
timing signal temporarily when there is an error condition in 
the encoded coarse offset information, or the edge 55 
information, as shown in FIG. 11. If an error is detected by 
the receiving office in the CRC, the edge bits, or a sudden 
change in the coarse offset value from a predicted value 
shows a loss of synchronization at the transmitting office, the 
office generates an error signal 252. The output of the 60 
decoder 126 is provided to one input of a multiplexer 254. 
The other input is provided by dividing down the BITS clock 
108 by eight hundred with divider 256 where the output of 
the divider is synchronized to valid rising edges of the 
regenerated signal according to well known techniques. The 65 
output of the divider 256 is coupled to the other input of the 
multiplexer 254 to provide a temporary backup version of 
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the regenerated timing signal. Therefore, whenever the 
office detects an error condition, the error signals 252 can 
select the temporary back up signal to provide the regener- 
ated timing signal 114. 

For propagation of synchronization throughout the 
network, each node not serving as a master clock source can 
both receive and regenerate the timing reference signal and 
also generate a timing reference signal and transmit the 
difference between that generated timing reference signal 
and the frame. Since each node in the network receives and 
regenerates the same synchronization information, the net- 
work architecture is essentially flat Further, timing loops arc 
eliminated due to the usage of such a flat architecture. 

Although a particular embodiment of the invention is 
disclosed, alternatives would be readily apparent to those of 
skill in the field. Different frequencies for signals arc, of 
course, appropriate for different networks such as OC-N or 
SDH where line clock frequencies arc integer multiples of 
51.84 MHz or 155 MHz, respectively. In fact, a 19.44 MHz 
clock, which is readily available in many implementations, 
may also be used, instead of the actual line clock, to measure 
the timing difference. Also, different protocols can be used 
for encoding and transmitting the timing difference signal. 
Instead of using counters to generate the timing difference 
various types of analog and digital phase detectors may be 
used. Alternatively, the regenerated timing signal could also 
be obtained through the use of a high precision numerically 
controlled oscillator controlled by a microprocessor using 
the coarse offset information to generate the timing refer- 
ence signal at the output of the oscillator. In addition, while 
the disclosed embodiments use the start of the frame as a 
reference for generating the timing difference, other specific 
times in the frame may also be used for generating the 
timing difference with the local timing reference signal. 

Further, instead of using the PRS as an original source for 
the synchronization signal to be distributed, other sources 
may be used such as the disciplined time scale generator 
disclosed in U.S. patent application Ser. No. 08/278,423 to 
Zarapetti, pages, 9-26 of which are incorporated herein by 
reference. By equipping occasional offices in the chain with 
such timescalc generators that are disciplined to a universal 
time scale such as GPS or LORAN, a highly synchronous 
network is established without requiring the expense of 
numerous PRS clocks or disciplined time scale generators at 
each office. Resort to the scope of the invention should be 
through the claims. 

We claim: 

1. A method of passing synchronization through a net- 
work comprised of a plurality of nodes communicating with 
each other at a predetermined frame rate, the nodes com- 
municating with each other through frames having pre- 
defined starts and with a line clock, the communication 
occurring by transmitting the frame and the line clock 
between nodes, the method comprising: 

generating at a first node a local timing reference signal at 
a frequency that is less than the frame rale; 

determining with the line clock at the first node the timing 
difference between a predetermined time of the frame 
and the local timing reference signal; and 

transmitting the timing difference to at least one other 
node in the network. 

2. The method of claim 1, wherein the method further 
comprises: 

determining the time of the frame and recovering the line 
clock at a second node coupled to the first node; 

regenerating the timing reference signal based upon the 
transmitted timing difference. 
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3. The method of claim 1, wherein the frequency of the 
local timing reference signal is selected to minimize the 
occurrence of the metastable states in successive cycles of 
the timing signal. 

4. The method of claim 1, wherein the net work is a 
SONET network and the SONET network frame includes an 
Fl byte, the transmission of information indicating each 
timing difference occurs over multiple frames in the Fl byte. 

5. A method for maintaining synchronization in at least a 
portion of a network of a plurality of nodes, the network 
having a reference timing signal, each node in the network 
communicating with at least one other node in the network 
through frames having frame timing and each node gener- 
ating a line clock and frame timing for communication with 
at least one other node, the method comprising: 

at at least one first node generating a local timing refer- 
ence signal based at least in part upon the network 
reference timing signal; 

measuring the difference in timing at the first node 
between the frame timing for at least some frames 
being transmitted and the local timing reference signal; 

transmitting in at least some frames to at least one second 
node from the first node the measured lime difference; 
and 

generating at the second node with the line clock, the 
frame timing and the transmitted measured time dif- 
ference a reconstruction of the local timing reference 
signal such that at least one reconstructed local timing 
reference signal is synchronized to the network refer- 
ence timing signal. 

6. An apparatus for aiding the distribution of synchroni- 
zation throughout a network comprising a plurality of nodes, 
each node generating a line clock having clock pulses at a 
predetermined frequency for transmitting information to 
another node and each node transmitting according to 
frames having a period based upon the timing of the frame 

- that is being transmitted, the apparatus further including: 
a clock generator providing a timing reference signal 
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12. The apparatus of claim 11, wherein the timing refer- 
ence signal is generated based upon the regenerated timing 
signal. 

13. The apparatus of claim 11, wherein the regenerated 
timing signal is the timing reference signal. 

14. An apparatus for aiding the distribution of synchro- 
nization throughout a network comprised a plurality of 
nodes, each node regenerating a line clock at a predeter- 
mined frequency for receiving information from another 
node and each node receiving data according to frames 
having a period based upon the timing of the frame that is 
being received, the network further including transmission 
of information from a node related to the timing difference 
based upon the line clock between a predetermined portion 
of the frame and a timing reference signal, the apparatus 
further including: 

means for regenerating the line clock; 

means for regenerating timing associated with the 

received frame; and 
means for reconstructing the timing reference signal from 

the received information, the regenerated line clock and 

the regenerated frame timing. 

15. The apparatus of claim 14, wherein the timing differ- 
ence is detected by counting the number of regenerated 
clock pulses from the point of the frame timing based upon 
the received information. 

16. The apparatus of claim 15, wherein the frequency of 
the timing reference signal is a relative prime when com- 
pared with the frame timing. 

17. The apparatus of claim 14, wherein the frame timing 
has a frequency of about 8 KHz and the reference timing 
signal has a frequency of about 7.72 KHz or an integral 
submultiple of 7.72 KHz. 

18. A method for generating synchronization throughout 
at least a part of a network based upon a master clock, each 
node of the at least part of the network having a node clock, 
whereby the phase timing of the node clock is synchronized 
to the timing of the master clock with network protocol 
including a predetermined timing relationship for the infor- 


having a period that is greater than the period of the 40 mation being transmitted, the method including: 


frame rate; and 
a timing difference detector detecting a timing difference 
from time to time between a specific time in the frame 
and the timing reference signal. 

7. The apparatus of claim 6, wherein the detected timing 45 
difference is transmitted to another node within the network. 

8. The apparatus of claim 6, wherein the timing difference 
is detected by counting the number of clock pulses between 
a fixed reference point in the timing reference signal and the 
start of a frame. 

9. The apparatus of claim 8, wherein the frequency of the 
timing reference signal is relatively prime when compared 
with the frame timing and the line clock pulse rate. 

10. The apparatus of claim 6, wherein the frame timing 
has a frequency of about 8 KHz and the timing reference 
signal has a frequency of about 7.72 KHz or an integral 
submultiple of 7.72 KHz. 

11. The apparatus of claim 6, wherein the node also 
regenerates a line clock signal received from a second node 
and regenerates a frame timing received from frames 
received from said second node, and the apparatus receives 
information from time to time indicating the difference in 
time between the start of a received frame and a timing 
reference signal at said second node, the apparatus further 
including: 

means responsive to the received timing difference for 
regenerating the timing signal at the first node. 
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measuring a timing difference between a reference signal 
and the predetermined timing relationship for the infor- 
mation being transmitted, the measurement being 
achieved with a predetermined granularity; 

transmitting the measurement to at least one other node 
where the measurement is received; 

synchronizing the clock of the node to the master clock 
using the received information, the timing relationship 
of the network and a filtering algorithm to provide a 
long term phase synchronization with the master clock 
at a resolution that is at least one order of magnitude 
smaller than the measuring granularity. 

19. The apparatus of claim 18, wherein the means for 
reconstruction comprises a counter counting a predeter- 
mined number of line clock pulses from a predetermined 
portion of the frame based upon the transmitted information 
to thereby generate a pulse for the timing reference signal. 

20. The method of claim 19, wherein the method further 
includes transmitting the measurement from a node having 
a master clock to a second node and the second node 
generating a timing signal in a phase lock relationship with 
the master clock based upon the transmitted information, the 
second node using the generated timing signal for making a 
timing difference measurement for transmission to a third 
node, the clock in the third node being maintained in 
synchronization with the first node based upon the trans- 
mitted timing information. 
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21. A method for transmit ling timing information 
throughout at least part of a network, the network transmit- 
ting information during predetermined timing slots and a 
propagation delay existing in the network in the transmis- 
sion of information between adjacent nodes, the method 5 
including: 

making a liming measurement based upon a timing ref- 
erence and a timing information being transmitted 
during a first timing slot; 

transmitting the timing measurement during a subsequent 10 
timing slot; 

recovering the timing measurement at a second node 
during the immediately subsequent timing slot such 
that the timing information at the second node is J5 
recovered two timing slots plus a propagation delay 
after the first timing slot. 

22. A method of passing synchronization through a net- 
work comprised of a plurality of nodes communicating with 
each other at a predetermined frame rate, the nodes com- ^ 
municating with each other through frames having pre- 
defined starts and with a line clock, the communication 
occurring by transmitting the frame and the line clock 
between nodes, the method comprising: 

generating at a first node a local timing reference signal; 23 
determining with the line clock at the first node the timing 
difference between a predetermined time of the frame 
being transmitted and the local timing reference signal; 
transmitting the timing difference to at least one other 
node in the network. 30 

23. The method of claim 22, wherein the method further 
comprises: 

determining the time of the frame and recovering the line 
clock at a second node coupled to the first node; 

regenerating the timing reference signal based upon the 
transmitted timing difference. 

24. The method of claim 22, wherein the frequency of the 
local timing reference signal is selected to minimize the 
occurrence of the metastable states in successive cycles of 

..... '40 

the timing signal. 

25. The method of claim 22, wherein the network is a 
SONET network and the SONET network frame includes an 
Fl byte, the transmission of information indicating each 
timing difference occurs over multiple frames in the Fl byte. 
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26. An apparatus for aiding the distribution of synchro- 
nization throughout a network comprising a plurality of 
nodes, each node generating a line clock having clock pulses 
at a predetermined frequency for transmitting information to 
another node and each node transmitting according to 
frames having a period based upon the timing of the frame 
that is being transmitted, the apparatus further including: 

a clock generator providing a timing reference signal; and 
a timing difference detector detecting a timing difference 
from time to time between a specific lime in the frame 
being transmitted and the timing reference signal. 

27. The apparatus of claim 26, wherein the detected 
liming difference is transmitted to another node within the 
network. 

28. The apparatus of claim 26, wherein the timing differ- 
ence is detected by counting the number of clock pulses 
between a fixed reference point in the timing reference 
signal and the start of a frame. 

29. The apparatus of claim 28, wherein the frequency of 
the timing reference signal is relatively prime when com- 
pared with the frame timing and the line clock pulse rate. 

30. The apparatus of claim 26, wherein the frame timing 
has a frequency of about 8 KHz and the timing reference 
signal has a frequency of about 7.72 KHz or an integral 
submultiple of 7.72 KHz. 

31. The apparatus of claim 26, wherein the node also 
regenerates a line clock signal received from a second node 
and regenerates a frame timing received from frames 
received from said second node, and the apparatus receives 
information from time to time indicating the difference in 
time between the start is a received frame and a timing 
reference signal at said second node, the apparatus further 
including: 

means responsive to the received timing difference for 
regenerating the timing signal at the first node. 

32. The apparatus of claim 31, wherein the timing refer- 
ence signal is generated based upon the regenerated timing 
signal. 

33. The apparatus of claim 31, wherein the regenerated 
timing signal is the timing reference signal. 
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